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PREFACE 


Here is your book, the one your thousands of letters have asked us 

to publish. It has taken us years to do, checking and rechecking countless 
recipes to bring you only the best, only the interesting, only the perfect. 
Now we can say, without a shadow of a doubt, that every single one of them, 
if you follow the directions to the letter, will work for you exactly as well 

as it did for us, even if you have never cooked before. 


— McCall's Cookbook (1963) 


THE PROCESS of preparing programs for a digital computer is especially attrac- 
tive, not only because it can be economically and scientifically rewarding, but 
also because it can be an aesthetic experience much like composing poetry or 
music. This book is the first volume of a multi-volume set of books that has been 
designed to train the reader in various skills that go into a programmer’s craft. 
The following chapters are not meant to serve as an introduction to computer 
programming; the reader is supposed to have had some previous experience. The 
prerequisites are actually very simple, but a beginner requires time and practice 
in order to understand the concept of a digital computer. The reader should 
possess: 
a) Some idea of how a stored-program digital computer works; not necessarily 
the electronics, rather the manner in which instructions can be kept in the 
machine’s memory and successively executed. 


b) An ability to put the solutions to problems into such explicit terms that a 
computer can “understand” them. (These machines have no common sense; 
they do exactly as they are told, no more and no less. This fact is the 
hardest concept to grasp when one first tries to use a computer.) 


c) Some knowledge of the most elementary computer techniques, such as loop- 
ing (performing a set of instructions repeatedly), the use of subroutines, and 
the use of indexed variables. 
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d) A little knowledge of common computer jargon— “memory,” “registers,” 
“bits,” “floating point,” “overflow,” “software.” Most words not defined in 
the text are given brief definitions in the index at the close of each volume. 


These four prerequisites can perhaps be summed up into the single requirement 
that the reader should have already written and tested at least, say, four pro- 
grams for at least one computer. 

I have tried to write this set of books in such a way that it will fill several 
needs. In the first place, these books are reference works that summarize the 
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knowledge that has been acquired in several important fields. In the second place, 
they can be used as textbooks for self-study or for college courses in the computer 
and information sciences. To meet both of these objectives, I have incorporated 
a large number of exercises into the text and have furnished answers for most 
of them. I have also made an effort to fill the pages with facts rather than with 
vague, general commentary. 

This set of books is intended for people who will be more than just casually 
interested in computers, yet it is by no means only for the computer specialist. 
Indeed, one of my main goals has been to make these programming techniques 
more accessible to the many people working in other fields who can make fruitful 
use of computers, yet who cannot afford the time to locate all of the necessary 
information that is buried in technical journals. 

We might call the subject of these books “nonnumerical analysis.” Comput- 
ers have traditionally been associated with the solution of numerical problems 
such as the calculation of the roots of an equation, numerical interpolation 
and integration, etc., but such topics are not treated here except in passing. 
Numerical computer programming is an extremely interesting and rapidly ex- 
panding field, and many books have been written about it. Since the early 
1960s, however, computers have been used even more often for problems in which 
numbers occur only by coincidence; the computer’s decision-making capabilities 
are being used, rather than its ability to do arithmetic. We have some use 
for addition and subtraction in nonnumerical problems, but we rarely feel any 
need for multiplication and division. Of course, even a person who is primarily 
concerned with numerical computer programming will benefit from a study of 
the nonnumerical techniques, for they are present in the background of numerical 
programs as well. 

The results of research in nonnumerical analysis are scattered throughout 
numerous technical journals. My approach has been to try to distill this vast 
literature by studying the techniques that are most basic, in the sense that they 
can be applied to many types of programming situations. I have attempted to 
coordinate the ideas into more or less of a “theory,” as well as to show how the 
theory applies to a wide variety of practical problems. 

Of course, “nonnumerical analysis” is a terribly negative name for this field 
of study; it is much better to have a positive, descriptive term that characterizes 
the subject. “Information processing” is too broad a designation for the material 
I am considering, and “programming techniques” is too narrow. Therefore I wish 
to propose analysis of algorithms as an appropriate name for the subject matter 
covered in these books. This name is meant to imply “the theory of the properties 
of particular computer algorithms.” 


The complete set of books, entitled The Art of Computer Programming, has 
the following general outline: 
Volume 1. Fundamental Algorithms 


Chapter 1. Basic Concepts 
Chapter 2. Information Structures 
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Volume 2. Seminumerical Algorithms 


Chapter 3. Random Numbers 
Chapter 4. Arithmetic 


Volume 3. Sorting and Searching 


Chapter 5. Sorting 
Chapter 6. Searching 


Volume 4. Combinatorial Algorithms 


Chapter 7. Combinatorial Searching 
Chapter 8. Recursion 


Volume 5. Syntactical Algorithms 


Chapter 9. Lexical Scanning 
Chapter 10. Parsing 


Volume 4 deals with such a large topic, it actually represents several separate 
books (Volumes 4A, 4B, and so on). Two additional volumes on more specialized 
topics are also planned: Volume 6, The Theory of Languages (Chapter 11); 
Volume 7, Compilers (Chapter 12). 

I started out in 1962 to write a single book with this sequence of chapters, 
but I soon found that it was more important to treat the subjects in depth rather 
than to skim over them lightly. The resulting length of the text has meant that 
each chapter by itself contains more than enough material for a one-semester 
college course; so it has become sensible to publish the series in separate volumes. 
I know that it is strange to have only one or two chapters in an entire book, but 
I have decided to retain the original chapter numbering in order to facilitate 
cross references. A shorter version of Volumes 1 through 5 is planned, intended 
specifically to serve as a more general reference and/or text for undergraduate 
computer courses; its contents will be a subset of the material in these books, 
with the more specialized information omitted. The same chapter numbering 
will be used in the abridged edition as in the complete work. 

The present volume may be considered as the “intersection” of the entire set, 
in the sense that it contains basic material that is used in all the other books. 
Volumes 2 through 5, on the other hand, may be read independently of each 
other. Volume 1 is not only a reference book to be used in connection with the 
remaining volumes; it may also be used in college courses or for self-study as a 
text on the subject of data structures (emphasizing the material of Chapter 2), 
or as a text on the subject of discrete mathematics (emphasizing the material of 
Sections 1.1, 1.2, 1.3.3, and 2.3.4), or as a text on the subject of machine-language 
programming (emphasizing the material of Sections 1.3 and 1.4). 

The point of view I have adopted while writing these chapters differs from 
that taken in most contemporary books about computer programming in that 
Iam not trying to teach the reader how to use somebody else’s software. I am 
concerned rather with teaching people how to write better software themselves. 
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My original goal was to bring readers to the frontiers of knowledge in every 
subject that was treated. But it is extremely difficult to keep up with a field 
that is economically profitable, and the rapid rise of computer science has made 
such a dream impossible. The subject has become a vast tapestry with tens of 
thousands of subtle results contributed by tens of thousands of talented people 
all over the world. Therefore my new goal has been to concentrate on “classic” 
techniques that are likely to remain important for many more decades, and to 
describe them as well as I can. In particular, I have tried to trace the history 
of each subject, and to provide a solid foundation for future progress. I have 
attempted to choose terminology that is concise and consistent with current 
usage. I have tried to include all of the known ideas about sequential computer 
programming that are both beautiful and easy to state. 


A few words are in order about the mathematical content of this set of books. 
The material has been organized so that persons with no more than a knowledge 
of high-school algebra may read it, skimming briefly over the more mathematical 
portions; yet a reader who is mathematically inclined will learn about many 
interesting mathematical techniques related to discrete mathematics. This dual 
level of presentation has been achieved in part by assigning ratings to each of the 
exercises so that the primarily mathematical ones are marked specifically as such, 
and also by arranging most sections so that the main mathematical results are 
stated before their proofs. The proofs are either left as exercises (with answers 
to be found in a separate section) or they are given at the end of a section. 

A reader who is interested primarily in programming rather than in the 
associated mathematics may stop reading most sections as soon as the math- 
ematics becomes recognizably difficult. On the other hand, a mathematically 
oriented reader will find a wealth of interesting material collected here. Much of 
the published mathematics about computer programming has been faulty, and 
one of the purposes of this book is to instruct readers in proper mathematical 
approaches to this subject. Since I profess to be a mathematician, it is my duty 
to maintain mathematical integrity as well as I can. 

A knowledge of elementary calculus will suffice for most of the mathematics 
in these books, since most of the other theory that is needed is developed herein. 
However, I do need to use deeper theorems of complex variable theory, probability 
theory, number theory, etc., at times, and in such cases I refer to appropriate 
textbooks where those subjects are developed. 


The hardest decision that I had to make while preparing these books con- 
cerned the manner in which to present the various techniques. The advantages of 
flow charts and of an informal step-by-step description of an algorithm are well 
known; for a discussion of this, see the article “Computer-Drawn Flowcharts” 
in the ACM Communications, Vol. 6 (September 1963), pages 555-563. Yet a 
formal, precise language is also necessary to specify any computer algorithm, 
and I needed to decide whether to use an algebraic language, such as ALGOL 
or FORTRAN, or to use a machine-oriented language for this purpose. Per- 
haps many of today’s computer experts will disagree with my decision to use a 
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machine-oriented language, but I have become convinced that it was definitely 
the correct choice, for the following reasons: 


a) A programmer is greatly influenced by the language in which programs are 
written; there is an overwhelming tendency to prefer constructions that are 
simplest in that language, rather than those that are best for the machine. 
By understanding a machine-oriented language, the programmer will tend 
to use a much more efficient method; it is much closer to reality. 


b) The programs we require are, with a few exceptions, all rather short, so with 
a suitable computer there will be no trouble understanding the programs. 

c) High-level languages are inadequate for discussing important low-level de- 
tails such as coroutine linkage, random number generation, multi-precision 
arithmetic, and many problems involving the efficient usage of memory. 


d) A person who is more than casually interested in computers should be well 
schooled in machine language, since it is a fundamental part of a computer. 


e) Some machine language would be necessary anyway as output of the software 
programs described in many of the examples. 


f) New algebraic languages go in and out of fashion every five years or so, while 

I am trying to emphasize concepts that are timeless. 

From the other point of view, I admit that it is somewhat easier to write programs 
in higher-level programming languages, and it is considerably easier to debug 
the programs. Indeed, I have rarely used low-level machine language for my 
own programs since 1970, now that computers are so large and so fast. Many 
of the problems of interest to us in this book, however, are those for which 
the programmer’s art is most important. For example, some combinatorial 
calculations need to be repeated a trillion times, and we save about 11.6 days 
of computation for every microsecond we can squeeze out of their inner loop. 
Similarly, it is worthwhile to put an additional effort into the writing of software 
that will be used many times each day in many computer installations, since the 
software needs to be written only once. 

Given the decision to use a machine-oriented language, which language 
should be used? I could have chosen the language of a particular machine X, 
but then those people who do not possess machine X would think this book is 
only for X-people. Furthermore, machine X probably has a lot of idiosyncrasies 
that are completely irrelevant to the material in this book yet which must be 
explained; and in two years the manufacturer of machine X will put out machine 
X +1 or machine 10X, and machine X will no longer be of interest to anyone. 

To avoid this dilemma, I have attempted to design an “ideal” computer 
with very simple rules of operation (requiring, say, only an hour to learn), which 
also resembles actual machines very closely. There is no reason why a student 
should be afraid of learning the characteristics of more than one computer; once 
one machine language has been mastered, others are easily assimilated. Indeed, 
serious programmers may expect to meet many different machine languages in 
the course of their careers. So the only remaining disadvantage of a mythical 
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machine is the difficulty of executing any programs written for it. Fortunately, 
that is not really a problem, because many volunteers have come forward to 
write simulators for the hypothetical machine. Such simulators are ideal for 
instructional purposes, since they are even easier to use than a real computer 
would be. 


I have attempted to cite the best early papers in each subject, together with 
a sampling of more recent work. When referring to the literature, I use standard 
abbreviations for the names of periodicals, except that the most commonly cited 
journals are abbreviated as follows: 


CACM = Communications of the Association for Computing Machinery 
JACM = Journal of the Association for Computing Machinery 

Comp. J. = The Computer Journal (British Computer Society) 

Math. Comp. = Mathematics of Computation 

AMM = American Mathematical Monthly 

SICOMP = SIAM Journal on Computing 

FOCS = IEEE Symposium on Foundations of Computer Science 

SODA = ACM-SIAM Symposium on Discrete Algorithms 

STOC = ACM Symposium on Theory of Computing 

Crelle = Journal fiir die reine und angewandte Mathematik 


As an example, “CACM 6 (1963), 555-563” stands for the reference given in a 
preceding paragraph of this preface. I also use “CMath” to stand for the book 
Concrete Mathematics, which is cited in the introduction to Section 1.2. 

Much of the technical content of these books appears in the exercises. When 
the idea behind a nontrivial exercise is not my own, I have attempted to give 
credit to the person who originated that idea. Corresponding references to the 
literature are usually given in the accompanying text of that section, or in the 
answer to that exercise, but in many cases the exercises are based on unpublished 
material for which no further reference can be given. 

I have, of course, received assistance from a great many people during the 
years I have been preparing these books, and for this I am extremely thankful. 
Acknowledgments are due, first, to my wife, Jill, for her infinite patience, for 
preparing several of the illustrations, and for untold further assistance of all 
kinds; secondly, to Robert W. Floyd, who contributed a great deal of his time 
towards the enhancement of this material during the 1960s. Thousands of other 
people have also provided significant help—it would take another book just 
to list their names! Many of them have kindly allowed me to make use of 
hitherto unpublished work. My research at Caltech and Stanford was gener- 
ously supported for many years by the National Science Foundation and the 
Office of Naval Research. Addison—Wesley has provided excellent assistance and 
cooperation ever since I began this project in 1962. The best way I know how 
to thank everyone is to demonstrate by this publication that their input has led 
to books that resemble what I think they wanted me to write. 
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Preface to the Third Edition 


After having spent ten years developing the TẸX and METAFONT systems for 
computer typesetting, I am now able to fulfill the dream that I had when I 
began that work, by applying those systems to The Art of Computer Program- 
ming. At last the entire text of this book has been captured inside my personal 
computer, in an electronic form that will make it readily adaptable to future 
changes in printing and display technology. The new setup has allowed me to 
make literally thousands of improvements that I’ve been wanting to incorporate 
for a long time. 

In this new edition I have gone over every word of the text, trying to retain 
the youthful exuberance of my original sentences while perhaps adding some 
more mature judgment. Dozens of new exercises have been added; dozens of old 
exercises have been given new and improved answers. 

@ The Art of Computer Programming is, however, still a work in progress. 

1 Therefore some parts of this book are headed by an “under construction” 
icon, to apologize for the fact that the material is not up-to-date. My files are 
bursting with important material that I plan to include in the final, glorious, 
fourth edition of Volume 1, perhaps 15 years from now; but I must finish 
Volumes 4 and 5 first, and I do not want to delay their publication any more 
than absolutely necessary. 


My efforts to extend and enhance these volumes have been enormously en- 
hanced since 1980 by the wise guidance of Addison—Wesley’s editor Peter Gordon. 
He has become not only my “publishing partner” but also a close friend, while 
continually nudging me to move in fruitful directions. Indeed, my interactions 
with dozens of Addison-Wesley people during more than three decades have 
been much better than any author deserves. The tireless support of managing 
editor John Fuller, whose meticulous attention to detail has maintained the 
highest standards of production quality in spite of frequent updates, has been 
particularly praiseworthy. 

Most of the hard work of preparing the new edition was accomplished by 
Phyllis Winkler and Silvio Levy, who expertly keyboarded and edited the text 
of the second edition, and by Jeffrey Oldham, who converted nearly all of the 
original illustrations to METAPOST format. I have corrected every error that 
alert readers detected in the second edition (as well as some mistakes that, alas, 
nobody noticed); and I have tried to avoid introducing new errors in the new 
material. However, I suppose some defects still remain, and I want to fix them 
as soon as possible. Therefore I will cheerfully award $2.56 to the first finder of 
each technical, typographical, or historical error. The webpage cited on page iv 
contains a current listing of all corrections that have been reported to me. 


Stanford, California D. E. K. 
April 1997 


Things have changed in the past two decades. 
— BILL GATES (1995) 
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Procedure for Reading 
This Set of Books 


. Begin reading this procedure, unless you have already begun to read it. 


Continue to follow the steps faithfully. (The general form of this procedure 
and its accompanying flow chart will be used throughout this book.) 


. Read the Notes on the Exercises, on pages xv—xvii. 
. Set N equal to 1. 
. Begin reading Chapter N. Do not read the quotations that appear at the 


beginning of the chapter. 


. Is the subject of the chapter interesting to you? If so, go to step 7; if not, 


go to step 6. 


. Is N < 2? If not, go to step 16; if so, scan through the chapter anyway. 


(Chapters 1 and 2 contain important introductory material and also a review 
of basic programming techniques. You should at least skim over the sections 
on notation and about MIX.) 


. Begin reading the next section of the chapter; if you have already reached 


the end of the chapter, however, go to step 16. 


. Is section number marked with “*”? If so, you may omit this section on 


first reading (it covers a rather specialized topic that is interesting but not 
essential); go back to step 7. 


. Are you mathematically inclined? If math is all Greek to you, go to step 11; 


otherwise proceed to step 10. 


Check the mathematical derivations made in this section (and report errors 
to the author). Go to step 12. 


. If the current section is full of mathematical computations, you had better 


omit reading the derivations. However, you should become familiar with the 
basic results of the section; they are usually stated near the beginning, or 
in slanted type right at the very end of the hard parts. 

Work the recommended exercises in this section in accordance with the hints 
given in the Notes on the Exercises (which you read in step 2). 

After you have worked on the exercises to your satisfaction, check your 
answers with the answer printed in the corresponding answer section at the 
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14. 
15. 
16. 


17. 
18. 


PROCEDURE FOR READING THIS SET OF BOOKS 


rear of the book (if any answer appears for that problem). Also read the 
answers to the exercises you did not have time to work. Note: In most cases 
it is reasonable to read the answer to exercise n before working on exercise 
n + 1, so steps 12-13 are usually done simultaneously. 

Are you tired? If not, go back to step 7. 

Go to sleep. Then, wake up, and go back to step 7. 

Increase N by one. If N = 3, 5, 7, 9, 11, or 12, begin the next volume of 
this set of books. 

If N is less than or equal to 12, go back to step 4. 

Congratulations. Now try to get your friends to purchase a copy of Volume 1 
and to start reading it. Also, go back to step 3. 


Woe be to him that reads but one book. 
— GEORGE HERBERT, Jacula Prudentum, 1144 (1640) 


Le défaut unique de tous les ouvrages 
c'est d'être trop longs. 


— VAUVENARGUES, Réflexions, 628 (1746) 


Books are a triviality. Life alone is great. 
— THOMAS CARLYLE, Journal (1839) 


NOTES ON THE EXERCISES 


THE EXERCISES in this set of books have been designed for self-study as well as 
for classroom study. It is difficult, if not impossible, for anyone to learn a subject 
purely by reading about it, without applying the information to specific problems 
and thereby being encouraged to think about what has been read. Furthermore, 
we all learn best the things that we have discovered for ourselves. Therefore the 
exercises form a major part of this work; a definite attempt has been made to 
keep them as informative as possible and to select problems that are enjoyable 
as well as instructive. 

In many books, easy exercises are found mixed randomly among extremely 
difficult ones. A motley mixture is, however, often unfortunate because readers 
like to know in advance how long a problem ought to take — otherwise they 
may just skip over all the problems. A classic example of such a situation is 
the book Dynamic Programming by Richard Bellman; this is an important, 
pioneering work in which a group of problems is collected together at the end 
of some chapters under the heading “Exercises and Research Problems,” with 
extremely trivial questions appearing in the midst of deep, unsolved problems. 
It is rumored that someone once asked Dr. Bellman how to tell the exercises 
apart from the research problems, and he replied, “If you can solve it, it is an 
exercise; otherwise it’s a research problem.” 

Good arguments can be made for including both research problems and 
very easy exercises in a book of this kind; therefore, to save the reader from 
the possible dilemma of determining which are which, rating numbers have been 
provided to indicate the level of difficulty. These numbers have the following 
general significance: 


Rating Interpretation 


00 An extremely easy exercise that can be answered immediately if the 
material of the text has been understood; such an exercise can almost 
always be worked “in your head.” 


10 A simple problem that makes you think over the material just read, but 
is by no means difficult. You should be able to do this in one minute at 
most; pencil and paper may be useful in obtaining the solution. 


20 An average problem that tests basic understanding of the text mate- 
rial, but you may need about fifteen or twenty minutes to answer it 
completely. 
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30 A problem of moderate difficulty and/or complexity; this one may 
involve more than two hours’ work to solve satisfactorily, or even more 
if the TV is on. 

40 Quite a difficult or lengthy problem that would be suitable for a term 
project in classroom situations. A student should be able to solve the 
problem in a reasonable amount of time, but the solution is not trivial. 


50 A research problem that has not yet been solved satisfactorily, as far 
as the author knew at the time of writing, although many people have 
tried. If you have found an answer to such a problem, you ought to 
write it up for publication; furthermore, the author of this book would 
appreciate hearing about the solution as soon as possible (provided that 
it is correct). 


By interpolation in this “logarithmic” scale, the significance of other rating 
numbers becomes clear. For example, a rating of 17 would indicate an exercise 
that is a bit simpler than average. Problems with a rating of 50 that are 
subsequently solved by some reader may appear with a 40 rating in later editions 
of the book, and in the errata posted on the Internet (see page iv). 

The remainder of the rating number divided by 5 indicates the amount of 
detailed work required. Thus, an exercise rated 24 may take longer to solve 
than an exercise that is rated 25, but the latter will require more creativity. All 
exercises with ratings of 46 or more are open problems for future research, rated 
according to the number of different attacks that they’ve resisted so far. 

The author has tried earnestly to assign accurate rating numbers, but it is 
difficult for the person who makes up a problem to know just how formidable it 
will be for someone else to find a solution; and everyone has more aptitude for 
certain types of problems than for others. It is hoped that the rating numbers 
represent a good guess at the level of difficulty, but they should be taken as 
general guidelines, not as absolute indicators. 

This book has been written for readers with varying degrees of mathematical 
training and sophistication; as a result, some of the exercises are intended only for 
the use of more mathematically inclined readers. The rating is preceded by an M 
if the exercise involves mathematical concepts or motivation to a greater extent 
than necessary for someone who is primarily interested only in programming 
the algorithms themselves. An exercise is marked with the letters “HM” if its 
solution necessarily involves a knowledge of calculus or other higher mathematics 
not developed in this book. An “HM” designation does not necessarily imply 
difficulty. 

Some exercises are preceded by an arrowhead, “>”; this designates prob- 
lems that are especially instructive and especially recommended. Of course, no 
reader/student is expected to work all of the exercises, so those that seem to 
be the most valuable have been singled out. (This distinction is not meant to 
detract from the other exercises!) Each reader should at least make an attempt 
to solve all of the problems whose rating is 10 or less; and the arrows may help 
to indicate which of the problems with a higher rating should be given priority. 
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Solutions to most of the exercises appear in the answer section. Please use 
them wisely; do not turn to the answer until you have made a genuine effort to 
solve the problem by yourself, or unless you absolutely do not have time to work 
this particular problem. After getting your own solution or giving the problem a 
decent try, you may find the answer instructive and helpful. The solution given 
will often be quite short, and it will sketch the details under the assumption 
that you have earnestly tried to solve it by your own means first. Sometimes the 
solution gives less information than was asked; often it gives more. It is quite 
possible that you may have a better answer than the one published here, or you 
may have found an error in the published solution; in such a case, the author 
will be pleased to know the details. Later printings of this book will give the 
improved solutions together with the solver’s name where appropriate. 

When working an exercise you may generally use the answers to previous 
exercises, unless specifically forbidden from doing so. The rating numbers have 
been assigned with this in mind; thus it is possible for exercise n + 1 to have a 
lower rating than exercise n, even though it includes the result of exercise n as 
a special case. 


Summary of codes: 00 Immediate 
10 Simple (one minute) 
20 Medium (quarter hour) 


> Recommended 30 Moderately hard 

M Mathematically oriented 40 Term project 

HM Requiring “higher math” 50 Research problem 
EXERCISES 


> 1. [00] What does the rating “M20” mean? 
2. [10] Of what value can the exercises in a textbook be to the reader? 
3. [14] Prove that 13? = 2197. Generalize your answer. [This is an example of a 
horrible kind of problem that the author has tried to avoid.] 
4. [HM45] Prove that when n is an integer, n > 2, the equation z” + y” = z” has 
no solution in positive integers x,y, zZ. 


We can face our problem. 
We can arrange such facts as we have 
with order and method. 


— HERCULE POIROT, in Murder on the Orient Express (1934) 
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CHAPTER ONE 


BASIC CONCEPTS 


Many persons who are not conversant with mathematical studies 
imagine that because the business of [Babbage’s Analytical Engine] is to 
give its results in numerical notation, the nature of its processes must 
consequently be arithmetical and numerical, rather than algebraical and 
analytical. This is an error. The engine can arrange and combine its 
numerical quantities exactly as if they were letters or any other general 
symbols; and in fact it might bring out its results in algebraical notation, 
were provisions made accordingly. 


— AUGUSTA ADA, Countess of Lovelace (1843) 


Practice yourself, for heaven’s sake, in little things; 
and thence proceed to greater. 


— EPICTETUS (Discourses IV.i) 


1.1. ALGORITHMS 


THE NOTION of an algorithm is basic to all of computer programming, so we 
should begin with a careful analysis of this concept. 

The word “algorithm” itself is quite interesting; at first glance it may look 
as though someone intended to write “logarithm” but jumbled up the first four 
letters. The word did not appear in Webster’s New World Dictionary as late as 
1957; we find only the older form “algorism” with its ancient meaning, the process 
of doing arithmetic using Arabic numerals. During the Middle Ages, abacists 
computed on the abacus and algorists computed by algorism. By the time of the 
Renaissance, the origin of this word was in doubt, and early linguists attempted 
to guess at its derivation by making combinations like algiros [painful] + arithmos 
[number]; others said no, the word comes from “King Algor of Castile.” Finally, 
historians of mathematics found the true origin of the word algorism: It comes 
from the name of a famous Persian textbook author, Abu ‘Abd Allah Muhammad 
ibn Musa al-Khwarizmi (c. 825) — literally, “Father of Abdullah, Mohammed, 
son of Moses, native of Khwarizm.” The Aral Sea in Central Asia was once 
known as Lake Khwarizm, and the Khwarizm region is located in the Amu River 
basin just south of that sea. Al-Khwarizmi wrote the celebrated Arabic text 
Kitab al-jabr wa’l-mugabala (“Rules of restoring and equating”); another word, 
“algebra,” stems from the title of that book, which was a systematic study of the 
solution of linear and quadratic equations. [For notes on al-Khwarizm1’s life and 
work, see H. Zemanek, Lecture Notes in Computer Science 122 (1981), 1-81.] 
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Gradually the form and meaning of algorism became corrupted; as ex- 
plained by the Oxford English Dictionary, the word “passed through many 
pseudo-etymological perversions, including a recent algorithm, in which it is 
learnedly confused” with the Greek root of the word arithmetic. This change 
from “algorism” to “algorithm” is not hard to understand in view of the fact 
that people had forgotten the original derivation of the word. An early German 
mathematical dictionary, Vollständiges mathematisches Lexicon (Leipzig: 1747), 
gave the following definition for the word Algorithmus: “Under this designation 
are combined the notions of the four types of arithmetic calculations, namely 
addition, multiplication, subtraction, and division.” The Latin phrase algorith- 
mus infinitesimalis was at that time used to denote “ways of calculation with 
infinitely small quantities, as invented by Leibniz.” 

By 1950, the word algorithm was most frequently associated with Euclid’s 
algorithm, a process for finding the greatest common divisor of two numbers 
that appears in Euclid’s Elements (Book 7, Propositions 1 and 2). It will be 
instructive to exhibit Euclid’s algorithm here: 


Algorithm E (Euclid’s algorithm). Given two positive integers m and n, find 
their greatest common divisor, that is, the largest positive integer that evenly 
divides both m and n. 


E1. [Find remainder.] Divide m by n and let r be the remainder. (We will have 
O<r<n.) 


E2. [Is it zero?| If r = 0, the algorithm terminates; n is the answer. 


E3. [Reduce.] Set m + n, n + r, and go back to step El. | 


Of course, Euclid did not present his algorithm in just this manner. The 
format above illustrates the style in which all of the algorithms throughout this 
book will be presented. 

Each algorithm we consider has been given an identifying letter (E in the 
preceding example), and the steps of the algorithm are identified by this letter 
followed by a number (E1, E2, E3). The chapters are divided into numbered 
sections; within a section the algorithms are designated by letter only, but when 
algorithms are referred to in other sections, the appropriate section number is 
attached. For example, we are now in Section 1.1; within this section Euclid’s 
algorithm is called Algorithm E, while in later sections it is referred to as 
Algorithm 1.1E. 

Each step of an algorithm, such as step El above, begins with a phrase in 
brackets that sums up as briefly as possible the principal content of that step. 
This phrase also usually appears in an accompanying flow chart, such as Fig. 1, 
so that the reader will be able to picture the algorithm more readily. 

After the summarizing phrase comes a description in words and symbols 
of some action to be performed or some decision to be made. Parenthesized 
comments, like the second sentence in step E1, may also appear. Comments are 
included as explanatory information about that step, often indicating certain 
invariant characteristics of the variables or the current goals. They do not specify 
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j 


E1. Find remainder 


E2. Is it zero? E3. Reduce 


Fig. 1. Flow chart for Algorithm E. 


actions that belong to the algorithm, but are meant only for the reader’s benefit 
as possible aids to comprehension. 

The arrow “ +” in step E3 is the all-important replacement operation, some- 
times called assignment or substitution: “m < n” means that the value of 
variable m is to be replaced by the current value of variable n. When Algorithm E 
begins, the values of m and n are the originally given numbers; but when it 
ends, those variables will have, in general, different values. An arrow is used 
to distinguish the replacement operation from the equality relation: We will 
not say, “Set m = n,” but we will perhaps ask, “Does m = n?” The “=” 
sign denotes a condition that can be tested, the “4—” sign denotes an action 
that can be performed. The operation of increasing n by one is denoted by 
“n 4+ n + 1” (read “n is replaced by n + 1” or “n gets n +1”). In general, 
“variable + formula” means that the formula is to be computed using the present 
values of any variables appearing within it; then the result should replace the 
previous value of the variable at the left of the arrow. Persons untrained in 
computer work sometimes have a tendency to say “n becomes n + 1” and to 
write “n — n + 1” for the operation of increasing n by one; this symbolism can 
only lead to confusion because of its conflict with standard conventions, and it 
should be avoided. 

Notice that the order of actions in step E3 is important: “Set m + n, 
n <r” is quite different from “Set n + r, m + n,” since the latter would imply 
that the previous value of n is lost before it can be used to set m. Thus the 
latter sequence is equivalent to “Set n + r, m 4+ r.” When several variables 
are all to be set equal to the same quantity, we can use multiple arrows; for 
example, “n + r, m + r” may be written “n 4 m + r.” To interchange the 
values of two variables, we can write “Exchange m + n”; this action could also 
be specified by using a new variable t and writing “Set t+ m, m + n, n & t.” 

An algorithm starts at the lowest-numbered step, usually step 1, and it 
performs subsequent steps in sequential order unless otherwise specified. In step 
E3, the imperative “go back to step E1” specifies the computational order in an 
obvious fashion. In step E2, the action is prefaced by the condition “If r = 0”; 
so if r Æ 0, the rest of that sentence does not apply and no action is specified. 
We might have added the redundant sentence, “If r 4 0, go on to step E3.” 

The heavy vertical line “|” appearing at the end of step E3 is used to 
indicate the end of an algorithm and the resumption of text. 

We have now discussed virtually all the notational conventions used in the 
algorithms of this book, except for a notation used to denote “subscripted” or 
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“indexed” items that are elements of an ordered array. Suppose we have n 
quantities, v1, V2,..., Un; instead of writing v; for the jth element, the notation 
v[j] is often used. Similarly, afi, 7] is sometimes used in preference to a doubly 
subscripted notation like aij. Sometimes multiple-letter names are used for 
variables, usually set in capital letters; thus TEMP might be the name of a variable 
used for temporarily holding a computed value, PRIME[K] might denote the Kth 
prime number, and so on. 

So much for the form of algorithms; now let us perform one. It should be 
mentioned immediately that the reader should not expect to read an algorithm 
as if it were part of a novel; such an attempt would make it pretty difficult to 
understand what is going on. An algorithm must be seen to be believed, and the 
best way to learn what an algorithm is all about is to try it. The reader should 
always take pencil and paper and work through an example of each algorithm 
immediately upon encountering it in the text. Usually the outline of a worked 
example will be given, or else the reader can easily conjure one up. This is a 
simple and painless way to gain an understanding of a given algorithm, and all 
other approaches are generally unsuccessful. 

Let us therefore work out an example of Algorithm E. Suppose that we are 
given m = 119 and n = 544; we are ready to begin, at step El. (The reader 
should now follow the algorithm as we give a play-by-play account.) Dividing 
m by n in this case is quite simple, almost too simple, since the quotient is zero 
and the remainder is 119. Thus, r + 119. We proceed to step E2, and since 
r #0 no action occurs. In step E3 we set m + 544, n + 119. It is clear that if 
m < n originally, the quotient in step E1 will always be zero and the algorithm 
will always proceed to interchange m and n in this rather cumbersome fashion. 
We could insert a new step at the beginning: 


EO. [Ensure m > n.] If m < n, exchange m © n. 


This would make no essential change in the algorithm, except to increase its 
length slightly, and to decrease its running time in about one half of all cases. 

Back at step E1, we find that 544/119 = 4+ 68/119, sor + 68. Again E2 is 
inapplicable, and at E3 we set m + 119, n + 68. The next round sets r + 51, 
and ultimately m + 68, n + 51. Next r + 17, and m + 51, n + 17. Finally, 
when 51 is divided by 17, we set r + 0, so at step E2 the algorithm terminates. 
The greatest common divisor of 119 and 544 is 17. 

So this is an algorithm. The modern meaning for algorithm is quite similar to 
that of recipe, process, method, technique, procedure, routine, rigmarole, except 
that the word “algorithm” connotes something just a little different. Besides 
merely being a finite set of rules that gives a sequence of operations for solving 
a specific type of problem, an algorithm has five important features: 


1) Finiteness. An algorithm must always terminate after a finite number of 
steps. Algorithm E satisfies this condition, because after step E1 the value of r 
is less than n; so if r Æ 0, the value of n decreases the next time step E1 is 
encountered. A decreasing sequence of positive integers must eventually termi- 
nate, so step E1 is executed only a finite number of times for any given original 
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value of n. Note, however, that the number of steps can become arbitrarily large; 
certain huge choices of m and n will cause step El to be executed more than a 
million times. 

(A procedure that has all of the characteristics of an algorithm except that it 
possibly lacks finiteness may be called a computational method. Euclid originally 
presented not only an algorithm for the greatest common divisor of numbers, but 
also a very similar geometrical construction for the “greatest common measure” 
of the lengths of two line segments; this is a computational method that does 
not terminate if the given lengths are incommensurable. Another example of a 
nonterminating computational method is a reactive process, which continually 
interacts with its environment.) 


2) Definiteness. Each step of an algorithm must be precisely defined; the ac- 
tions to be carried out must be rigorously and unambiguously specified for each 
case. The algorithms of this book will hopefully meet this criterion, but they 
are specified in the English language, so there is a possibility that the reader 
might not understand exactly what the author intended. To get around this 
difficulty, formally defined programming languages or computer languages are 
designed for specifying algorithms, in which every statement has a very definite 
meaning. Many of the algorithms of this book will be given both in English 
and in a computer language. An expression of a computational method in a 
computer language is called a program. 

In Algorithm E, the criterion of definiteness as applied to step E1 means that 
the reader is supposed to understand exactly what it means to divide m by n 
and what the remainder is. In actual fact, there is no universal agreement about 
what this means if m and n are not positive integers; what is the remainder of 
—8 divided by —7? What is the remainder of 59/13 divided by zero? Therefore 
the criterion of definiteness means we must make sure that the values of m and n 
are always positive integers whenever step E1 is to be executed. This is initially 
true, by hypothesis; and after step El, r is a nonnegative integer that must be 
nonzero if we get to step E3. So m and n are indeed positive integers as required. 


3) Input. An algorithm has zero or more inputs: quantities that are given to it 
initially before the algorithm begins, or dynamically as the algorithm runs. These 
inputs are taken from specified sets of objects. In Algorithm E, for example, there 
are two inputs, namely m and n, both taken from the set of positive integers. 


4) Output. An algorithm has one or more outputs: quantities that have a 
specified relation to the inputs. Algorithm E has one output, namely n in step E2, 
the greatest common divisor of the two inputs. 

(We can easily prove that this number is indeed the greatest common divisor, 
as follows. After step E1, we have 


m= qn +r, 


for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case 
n is the greatest common divisor of m and n. If r 4 0, note that any number 
that divides both m and n must divide m — qn = r, and any number that divides 
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both n and r must divide gn +r = m; so the set of common divisors of m and n 
is the same as the set of common divisors of n and r. In particular, the greatest 
common divisor of m and n is the same as the greatest common divisor of n 
and r. Therefore step E3 does not change the answer to the original problem.) 


5) Effectiveness. An algorithm is also generally expected to be effective, in the 
sense that its operations must all be sufficiently basic that they can in principle 
be done exactly and in a finite length of time by someone using pencil and 
paper. Algorithm E uses only the operations of dividing one positive integer 
by another, testing if an integer is zero, and setting the value of one variable 
equal to the value of another. These operations are effective, because integers 
can be represented on paper in a finite manner, and because there is at least 
one method (the “division algorithm”) for dividing one by another. But the 
same operations would not be effective if the values involved were arbitrary real 
numbers specified by an infinite decimal expansion, nor if the values were the 
lengths of physical line segments (which cannot be specified exactly). Another 
example of a noneffective step is, “If 4 is the largest integer n for which there is 
a solution to the equation w” + x” + y” = z” in positive integers w, x, y, and z, 
then go to step E4.” Such a statement would not be an effective operation until 
someone successfully constructs an algorithm to determine whether 4 is or is not 
the largest integer with the stated property. 


Let us try to compare the concept of an algorithm with that of a cookbook 
recipe. A recipe presumably has the qualities of finiteness (although it is said 
that a watched pot never boils), input (eggs, flour, etc.), and output (TV dinner, 
etc.), but it notoriously lacks definiteness. There are frequent cases in which a 
cook’s instructions are indefinite: “Add a dash of salt.” A “dash” is defined 
to be “less than 1/g teaspoon,” and salt is perhaps well enough defined; but 
where should the salt be added—on top? on the side? Instructions like “toss 
lightly until mixture is crumbly” or “warm cognac in small saucepan” are quite 
adequate as explanations to a trained chef, but an algorithm must be specified 
to such a degree that even a computer can follow the directions. Nevertheless, 
a computer programmer can learn much by studying a good recipe book. (The 
author has in fact barely resisted the temptation to name the present volume 
“The Programmer’s Cookbook.” Perhaps someday he will attempt a book called 
“Algorithms for the Kitchen.”) 

We should remark that the finiteness restriction is not really strong enough 
for practical use. A useful algorithm should require not only a finite number 
of steps, but a very finite number, a reasonable number. For example, there is 
an algorithm that determines whether or not the game of chess can always be 
won by White if no mistakes are made (see exercise 2.2.3-28). That algorithm 
can solve a problem of intense interest to thousands of people, yet it is a safe 
bet that we will never in our lifetimes know the answer; the algorithm requires 
fantastically large amounts of time for its execution, even though it is finite. See 
also Chapter 8 for a discussion of some finite numbers that are so large as to 
actually be beyond comprehension. 
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In practice we not only want algorithms, we want algorithms that are good 
in some loosely defined aesthetic sense. One criterion of goodness is the length 
of time taken to perform the algorithm; this can be expressed in terms of the 
number of times each step is executed. Other criteria are the adaptability of the 
algorithm to different kinds of computers, its simplicity and elegance, etc. 

We often are faced with several algorithms for the same problem, and we 
must decide which is best. This leads us to the extremely interesting and 
all-important field of algorithmic analysis: Given an algorithm, we want to 
determine its performance characteristics. 

For example, let’s consider Euclid’s algorithm from this point of view. Sup- 
pose we ask the question, “Assuming that the value of n is known but m is 
allowed to range over all positive integers, what is the average number of times, 
Tn, that step E1 of Algorithm E will be performed?” In the first place, we need 
to check that this question does have a meaningful answer, since we are trying 
to take an average over infinitely many choices for m. But it is evident that 
after the first execution of step E1 only the remainder of m after division by n is 
relevant. So all we must do to find Tn is to try the algorithm for m = 1, m = 2, 
..., M = n, count the total number of times step El has been executed, and 
divide by n. 

Now the important question is to determine the nature of Tn; is it approxi- 
mately equal to in, or y/n, for instance? As a matter of fact, the answer to this 
question is an extremely difficult and fascinating mathematical problem, not yet 
completely resolved, which is examined in more detail in Section 4.5.3. For large 
values of n it is possible to prove that T, is approximately (12(In 2)/x?) Inn, 
that is, proportional to the natural logarithm of n, with a constant of propor- 
tionality that might not have been guessed offhand! For further details about 
Euclid’s algorithm, and other ways to calculate the greatest common divisor, see 
Section 4.5.2. 

Analysis of algorithms is the name the author likes to use to describe in- 
vestigations such as this. The general idea is to take a particular algorithm and 
to determine its quantitative behavior; occasionally we also study whether or 
not an algorithm is optimal in some sense. The theory of algorithms is another 
subject entirely, dealing primarily with the existence or nonexistence of effective 
algorithms to compute particular quantities. 

So far our discussion of algorithms has been rather imprecise, and a mathe- 
matically oriented reader is justified in thinking that the preceding commentary 
makes a very shaky foundation on which to erect any theory about algorithms. 
We therefore close this section with a brief indication of one method by which the 
concept of algorithm can be firmly grounded in terms of mathematical set theory. 
Let us formally define a computational method to be a quadruple (Q, I, Q, f), 
in which Q is a set containing subsets J and 2, and f is a function from Q 
into itself. Furthermore f should leave 2 pointwise fixed; that is, f(q) should 
equal q for all elements q of 2. The four quantities Q, I, 2, f are intended 
to represent respectively the states of the computation, the input, the output, 
and the computational rule. Each input x in the set I defines a computational 
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Sequence, To, £1, T2, ..., aS follows: 
zo =T and Ceti = f(£k) for k>0. (1) 


The computational sequence is said to terminate in k steps if k is the smallest 
integer for which x, is in Q, and in this case it is said to produce the output 
zk from z. (Notice that if 2, is in Q, so is £k+1, because £k+1 = £k in such 
a case.) Some computational sequences may never terminate; an algorithm is a 
computational method that terminates in finitely many steps for all x in I. 

Algorithm E may, for example, be formalized in these terms as follows: Let Q 
be the set of all singletons (n), all ordered pairs (m, n), and all ordered quadruples 
(m,n,7r,1), (m,n,r,2), and (m,n, p,3), where m, n, and p are positive integers 
and r is a nonnegative integer. Let I be the subset of all pairs (m,n) and let Q 
be the subset of all singletons (n). Let f be defined as follows: 


f((m,n)) = (m,n,0, 1); f((n)) = (n); 
f((m, n, T, 1)) = (m, n, remainder of m divided by n, 2); 
f(m, n,r, 2)) =(n) if r=0, (m,n,r,3) otherwise; 
f((m,n,p,3)) = (n, p, p, 1). 


The correspondence between this notation and Algorithm E is evident. 

This formulation of the concept of an algorithm does not include the re- 
striction of effectiveness mentioned earlier. For example, Q might denote infinite 
sequences that are not computable by pencil and paper methods, or f might 
involve operations that mere mortals cannot always perform. If we wish to 
restrict the notion of algorithm so that only elementary operations are involved, 
we can place restrictions on Q, I, 2, and f, for example as follows: Let A be 
a finite set of letters, and let A* be the set of all strings on A (the set of all 
ordered sequences 212%2...%,, where n > 0 and z; is in A for 1 < j < n). The 
idea is to encode the states of the computation so that they are represented by 
strings of A*. Now let N be a nonnegative integer and let Q be the set of all 
(o, j), where ø is in A* and j is an integer, 0 < j < N; let I be the subset of Q 
with 7 = 0 and let 2 be the subset with j = N. If 0 and o are strings in A*, we 
say that 0 occurs in ø if ø has the form aw for strings a and w. To complete 
our definition, let f be a function of the following type, defined by the strings 
0j, Qj and the integers aj, bj for 0 <j < N: 


(2) 


(lo, j))=(0,a;) if 0j does not occur in g; 
F(a, i)) =(aġjw,bj) if a is the shortest possible string for which o = a0j;w; 
f((o,N)) = (0, N). (3) 
Every step of such a computational method is clearly effective, and expe- 
rience shows that pattern-matching rules of this kind are also powerful enough 
to do anything we can do by hand. There are many other essentially equivalent 


ways to formulate the concept of an effective computational method (for example, 
using Turing machines). The formulation above is virtually the same as that 
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given by A. A. Markov in his book The Theory of Algorithms [Trudy Mat. Inst. 
Akad. Nauk 42 (1954), 1-376], later revised and enlarged by N. M. Nagorny 
(Moscow: Nauka, 1984; English edition, Dordrecht: Kluwer, 1988). 


EXERCISES 


1. [10] The text showed how to interchange the values of variables m and n, using 
the replacement notation, by setting t 4+ m, m + n, n + t. Show how the values of 
four variables (a,b,c,d) can be rearranged to (b,c, d,a) by a sequence of replacements. 
In other words, the new value of a is to be the original value of b, etc. Try to use the 
minimum number of replacements. 


2. [15] Prove that m is always greater than n at the beginning of step E1, except 
possibly the first time this step occurs. 

3. [20] Change Algorithm E (for the sake of efficiency) so that all trivial replacement 
operations such as “m + n” are avoided. Write this new algorithm in the style of 
Algorithm E, and call it Algorithm F. 


4. [16] What is the greatest common divisor of 2166 and 6099? 


5. [12] Show that the “Procedure for Reading This Set of Books” that appears after 
the preface actually fails to be a genuine algorithm on at least three of our five counts! 
Also mention some differences in format between it and Algorithm E. 


6. [20] What is Ts, the average number of times step E1 is performed when n = 5? 


7. [M21] Suppose that m is known and n is allowed to range over all positive integers; 
let Um be the average number of times that step E1 is executed in Algorithm E. Show 
that Um is well defined. Is Um in any way related to Tm? 


8. [M25] Give an “effective” formal algorithm for computing the greatest common 
divisor of positive integers m and n, by specifying 0;, j, aj, bj as in Eqs. (3). Let the 
input be represented by the string ab”, that is, m a’s followed by n b’s. Try to make 
your solution as simple as possible. [Hint: Use Algorithm E, but instead of division in 
step E1, set r + |m — n|, n © min(m, n).] 

9. [M30] Suppose that Cy = (Q1, 41, 21, fi) and C2 = (Q2, I2, 22, f2) are computa- 
tional methods. For example, Cı might stand for Algorithm E as in Eqs. (2), except 
that m and n are restricted in magnitude, and C2 might stand for a computer program 
implementation of Algorithm E. (Thus Q2 might be the set of all states of the machine, 
i.e., all possible configurations of its memory and registers; f2 might be the definition 
of single machine actions; and Iz might be the set of initial states, each including the 
program that determines the greatest common divisor as well as the particular values 
of m and n.) 

Formulate a set-theoretic definition for the concept “C2 is a representation of C1” 
or “C2 simulates C1.” This is to mean intuitively that any computation sequence of C1 
is mimicked by C2, except that C2 might take more steps in which to do the computation 
and it might retain more information in its states. (We thereby obtain a rigorous 
interpretation of the statement, “Program X is an implementation of Algorithm Y.”) 
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1.2. MATHEMATICAL PRELIMINARIES 


IN THIS SECTION we shall investigate the mathematical notations that occur 
throughout The Art of Computer Programming, and we’ll derive several basic 
formulas that will be used repeatedly. Even a reader not concerned with the 
more complex mathematical derivations should at least become familiar with 
the meanings of the various formulas, so as to be able to use the results of the 
derivations. 

Mathematical notation is used for two main purposes in this book: to 
describe portions of an algorithm, and to analyze the performance character- 
istics of an algorithm. The notation used in descriptions of algorithms is quite 
simple, as explained in the previous section. When analyzing the performance 
of algorithms, we need to use other more specialized notations. 

Most of the algorithms we will discuss are accompanied by mathematical 
calculations that determine the speed at which the algorithm may be expected 
to run. These calculations draw on nearly every branch of mathematics, and a 
separate book would be necessary to develop all of the mathematical concepts 
that are used in one place or another. However, the majority of the calculations 
can be carried out with a knowledge of college algebra, and the reader with a 
knowledge of elementary calculus will be able to understand nearly all of the 
mathematics that appears. Sometimes we will need to use deeper results of 
complex variable theory, group theory, number theory, probability theory, etc.; 
in such cases the topic will be explained in an elementary manner, if possible, or 
a reference to other sources of information will be given. 

The mathematical techniques involved in the analysis of algorithms usually 
have a distinctive flavor. For example, we will quite often find ourselves working 
with finite summations of rational numbers, or with the solutions to recurrence 
relations. Such topics are traditionally given only a light treatment in mathe- 
matics courses, and so the following subsections are designed not only to give a 
thorough drilling in the use of the notations to be defined but also to illustrate 
in depth the types of calculations and techniques that will be most useful to us. 


Important note: Although the following subsections provide a rather extensive 
training in the mathematical skills needed in connection with the study of com- 
puter algorithms, most readers will not see at first any very strong connections 
between this material and computer programming (except in Section 1.2.1). The 
reader may choose to read the following subsections carefully, with implicit faith 
in the author’s assertion that the topics treated here are indeed very relevant; but 
it is probably preferable, for motivation, to skim over this section lightly at first, 
and (after seeing numerous applications of the techniques in future chapters) 
return to it later for more intensive study. If too much time is spent studying 
this material when first reading the book, a person might never get on to the 
computer programming topics! However, each reader should at least become 
familiar with the general contents of these subsections, and should try to solve a 
few of the exercises, even on first reading. Section 1.2.10 should receive particular 
attention, since it is the point of departure for most of the theoretical material 
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developed later. Section 1.3, which follows 1.2, abruptly leaves the realm of 
“pure mathematics” and enters into “pure computer programming.” 

An expansion and more leisurely presentation of much of the following 
material can be found in the book Concrete Mathematics by Graham, Knuth, 
and Patashnik, second edition (Reading, Mass.: Addison-Wesley, 1994). That 
book will be called simply CMath when we need to refer to it later. 


1.2.1. Mathematical Induction 


Let P(n) be some statement about the integer n; for example, P(n) might be 
“n times (n + 3) is an even number,” or “if n > 10, then 2” > n°.” Suppose we 
want to prove that P(n) is true for all positive integers n. An important way to 
do this is: 
a) Give a proof that P(1) is true. 
b) Give a proof that “if all of P(1), P(2),...,P(n) are true, then P(n + 1) is 
also true”; this proof should be valid for any positive integer n. 
As an example, consider the following series of equations, which many people 
have discovered independently since ancient times: 
1=17, 
1+3=2?, 
1+34+5=3?, 
14+3454+7=4, 
14+3+5+7+9=5". (1) 


We can formulate the general property as follows: 
1+3+ + (2n-1) =n’. (2) 


Let us, for the moment, call this equation P(n); we wish to prove that P(n) is 
true for all positive n. Following the procedure outlined above, we have: 
a) “P(1) is true, since 1 = 1°.” 
b) “Ifall of P(1),..., P(n) are true, then, in particular, P(n) is true, so Eq. (2) 
holds; adding 2n + 1 to both sides we obtain 


14+3+---+(2n—1)+ (2n+1) =n? +2n4+1= (n4+1)?, 


which proves that P(n + 1) is also true.” 


We can regard this method as an algorithmic proof procedure. In fact, the 
following algorithm produces a proof of P(n) for any positive integer n, assuming 
that steps (a) and (b) above have been worked out: 


Algorithm I (Construct a proof). Given a positive integer n, this algorithm 
will output a proof that P(n) is true. 


I1. [Prove P(1).] Set k + 1, and, according to (a), output a proof of P(1). 
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I2. |k = n?] If k = n, terminate the algorithm; the required proof has been 
output. 

I3. [Prove P(k + 1).] According to (b), output a proof that “If all of P(1),..., 
P(k) are true, then P(k +1) is true.” Also output “We have already proved 
P(1),...,P(k); hence P(k + 1) is true.” 

T4. [Increase k.] Increase k by 1 and go to step I2. JJ 


——> Il. Prove P(1) I2. k=n? No > 13. Prove P(k +1) ~> I4. Increase k 


Fig. 2. Algorithm I: Mathematical induction. 


Since this algorithm clearly presents a proof of P(n), for any given n, the 
proof technique consisting of steps (a) and (b) is logically valid. It is called proof 
by mathematical induction. 

The concept of mathematical induction should be distinguished from what 
is usually called inductive reasoning in science. A scientist takes specific observa- 
tions and creates, by “induction,” a general theory or hypothesis that accounts 
for these facts; for example, we might observe the five relations in (1), above, 
and formulate (2). In this sense, induction is no more than our best guess about 
the situation; mathematicians would call it an empirical result or a conjecture. 

Another example will be helpful. Let p(n) denote the number of partitions 
of n, that is, the number of different ways to write n as a sum of positive integers, 
disregarding order. Since 5 can be partitioned in exactly seven ways, 


1+1+1+1+1=2+1+1+1=2+2+1=3+1+1=3+2=4+1=5, 
we have p(5) = 7. In fact, it is easy to establish the first few values, 


p(1)=1, p(2)=2, p(3)=3, p(4)=5, p(5)=7. 


At this point we might tentatively formulate, by induction, the hypothesis that 
the sequence p(2), p(3), ... runs through the prime numbers. To test this 
hypothesis, we proceed to calculate p(6) and behold! p(6) = 11, confirming 
our conjecture. 

[Unfortunately, p(7) turns out to be 15, spoiling everything, and we must 
try again. The numbers p(n) are known to be quite complicated, although 
S. Ramanujan succeeded in guessing and proving many remarkable things about 
them. For further information, see G. H. Hardy, Ramanujan (London: Cam- 
bridge University Press, 1940), Chapters 6 and 8. See also Section 7.2.1.4.] 

Mathematical induction is quite different from induction in the sense just 
explained. It is not just guesswork, but a conclusive proof of a statement; indeed, 
it is a proof of infinitely many statements, one for each n. It has been called 
“induction” only because one must first decide somehow what is to be proved, 
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before one can apply the technique of mathematical induction. Henceforth in 
this book we shall use the word induction only when we wish to imply proof by 
mathematical induction. 

There is a geometrical way to prove Eq. (2). 
Figure 3 shows, for n = 6, n? cells broken into 11 
groups of 1 +3 +--+- + (2n — 1) cells. However, in 
the final analysis, this picture can be regarded as a 
“proof” only if we show that the construction can 7 
be carried out for all n, and such a demonstration 
is essentially the same as a proof by induction. 

Our proof of Eq. (2) used only a special case 3 

of (b); we merely showed that the truth of P(n) 
implies the truth of P(n+1). This is an important 
simple case that arises frequently, but our next Fig. 3. The sum of odd 
example illustrates the power of the method a little numbers is a square. 
more. We define the Fibonacci sequence Fo, Fi, 
Fə, ... by the rule that Fo = 0, Fı = 1, and every further term is the sum 
of the preceding two. Thus the sequence begins 0, 1, 1, 2, 3, 5, 8, 13, ...; we 
will investigate it in detail in Section 1.2.8. We will now prove that if ¢ is the 
number (1 + v5 )/2 we have 


1 


Lag (3) 


for all positive integers n. Call this formula P(n). 

If n = 1, then F; = 1 = ¢° = ¢'"1, so step (a) has been done. For step (b) 
we notice first that P(2) is also true, since Fy = 1 < 1.6 < ọ! = ¢?"1. Now, if all 
of P(1), P(2),..., P(n) are true and n > 1, we know in particular that P(n—1) 
and P(n) are true; so F,_,; < 6"~? and Fp < "1. Adding these inequalities, 
we get 


Fai = Fait Fh < P? +¢" (= 2(1+ ¢). (4) 


The important property of the number ¢, indeed the reason we chose this number 
for this problem in the first place, is that 


1+¢=¢. (5) 


Plugging (5) into (4) gives Fr41 < 6", which is P(n +1). So step (b) has 
been done, and (3) has been proved by mathematical induction. Notice that we 
approached step (b) in two different ways here: We proved P(n+1) directly when 
n = 1, and we used an inductive method when n > 1. This was necessary, since 
when n = 1 our reference to P(n — 1) = P(0) would not have been legitimate. 

Mathematical induction can also be used to prove things about algorithms. 
Consider the following generalization of Euclid’s algorithm. 


Algorithm E (E£ztended Euclid’s algorithm). Given two positive integers m 
and n, we compute their greatest common divisor d, and we also compute two 
not-necessarily-positive integers a and b such that am + bn = d. 


E1. [Initialize] Seta’ + b + 1, a | b + 0, cm, den. 
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E2. [Divide.| Let q and r be the quotient and remainder, respectively, of c 
divided by d. (We have c= qd+rand0<r<d.) 

E3. [Remainder zero?] If r = 0, the algorithm terminates; we have in this case 
am + bn = d as desired. 

E4. [Recycle] Set c + d, d + r, t + da', ad' + a, a + t— qa, t | b, b + b 
b + t — qb, and go back to E2. J 


? 


If we suppress the variables a, b, a’, and b’ from this algorithm and use m 
and n for the auxiliary variables c and d, we have our old algorithm, 1.1E. The 
new version does a little more, by determining the coefficients a and b. Suppose 
that m = 1769 and n = 551; we have successively (after step E2): 


a! a b' b ë d q r 
1 0 0 1 1769 551 3 116 
0 1 1 —3 551 116 4 87 
1 —4 —3 13 116 87 1 29 

—4 5 13 —16 87 29 3 0 


The answer is correct: 5 x 1769 — 16 x 551 = 8845 — 8816 = 29, the greatest 
common divisor of 1769 and 551. 

The problem is to prove that this algorithm works properly, for all m and n. 
We can try to apply the method of mathematical induction by letting P(n) be 
the statement “Algorithm E works for n and all integers m.” However, that 
approach doesn’t work out so easily, and we need to prove some extra facts. 
After a little study, we find that something must be proved about a, b, a’, and b’, 
and the appropriate fact is that the equalities 


am+Un=c, am+bn=d (6) 


always hold whenever step E2 is executed. We may prove these equalities directly 
by observing that they are certainly true the first time we get to E2, and that 
step E4 does not change their validity. (See exercise 6.) 

Now we are ready to show that Algorithm E is valid, by induction on n: If 
m is a multiple of n, the algorithm obviously works properly, since we are done 
immediately at E3 the first time. This case always occurs when n = 1. The 
only case remaining is when n > 1 and m is not a multiple of n. In such a 
case, the algorithm proceeds to set c+ n, d« r after the first execution, and 
since r < n, we may assume by induction that the final value of d is the gcd 
of n and r. By the argument given in Section 1.1, the pairs {m,n} and {n,r} 
have the same common divisors, and, in particular, they have the same greatest 
common divisor. Hence d is the gcd of m and n, and am + bn = d by (6). 

The italicized phrase in the proof above illustrates the conventional lan- 
guage that is so often used in an inductive proof: When doing part (b) of the 
construction, rather than saying “We will now assume P(1), P(2),..., P(n), and 
with this assumption we will prove P(n+1),” we often say simply “We will now 
prove P(n); we may assume by induction that P(k) is true whenever 1 < k < n.” 
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Fig. 4. Flow chart for Algorithm E, labeled with assertions that prove the validity of 
the algorithm. 


If we examine this argument very closely and change our viewpoint slightly, 
we can envision a general method applicable to proving the validity of any 
algorithm. The idea is to take a flow chart for some algorithm and to label 
each of the arrows with an assertion about the current state of affairs at the 
time the computation traverses that arrow. See Fig. 4, where the assertions 
have been labeled A1, A2, ..., A6. (All of these assertions have the additional 
stipulation that the variables are integers; this stipulation has been omitted to 
save space.) A1 gives the initial assumptions upon entry to the algorithm, and 
A4 states what we hope to prove about the output values a, b, and d. 

The general method consists of proving, for each box in the flow chart, that 


if an assertion attached to any arrow leading into the box is true 
before the operation in that box is performed, then all of the 
assertions on relevant arrows leading away from the box are true 
after the operation. 


(7) 


Thus, for example, we must prove that either A2 or A6 before E2 implies A3 
after E2. (In this case A2 is a stronger statement than A6; that is, A2 implies 
A6. So we need only prove that A6 before E2 implies A3 after. Notice that the 
condition d > 0 is necessary in A6 just to prove that operation E2 even makes 
sense.) It is also necessary to show that A3 and r = 0 implies A4; that A3 and 
r #0 implies A5; etc. Each of the required proofs is very straightforward. 
Once statement (7) has been proved for each box, it follows that all assertions 
are true during any execution of the algorithm. For we can now use induction 
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on the number of steps of the computation, in the sense of the number of arrows 
traversed in the flow chart. While traversing the first arrow, the one leading from 
“Start”, the assertion A1 is true since we always assume that our input values 
meet the specifications; so the assertion on the first arrow traversed is correct. 
If the assertion that labels the nth arrow is true, then by (7) the assertion that 
labels the (n + 1)st arrow is also true. 

Using this general method, the problem of proving that a given algorithm 
is valid evidently consists mostly of inventing the right assertions to put in the 
flow chart. Once this inductive leap has been made, it is pretty much routine to 
carry out the proofs that each assertion leading into a box logically implies each 
assertion leading out. In fact, it is pretty much routine to invent the assertions 
themselves, once a few of the difficult ones have been discovered; thus it is very 
simple in our example to write out essentially what A2, A3, and A5 must be, 
if only A1, A4, and A6 are given. In our example, assertion A6 is the creative 
part of the proof; all the rest could, in principle, be supplied mechanically. Hence 
no attempt has been made to give detailed formal proofs of the algorithms that 
follow in this book, at the level of detail found in Fig. 4. It suffices to state 
the key inductive assertions. Those assertions either appear in the discussion 
following an algorithm or they are given as parenthetical remarks in the text of 
the algorithm itself. 

This approach to proving the correctness of algorithms has another aspect 
that is even more important: It mirrors the way we understand an algorithm. 
Recall that in Section 1.1 the reader was cautioned not to expect to read an 
algorithm like part of a novel; one or two trials of the algorithm on some sample 
data were recommended. This was done expressly because an example run- 
through of the algorithm helps a person formulate the various assertions mentally. 
It is the contention of the author that we really understand why an algorithm is 
valid only when we reach the point that our minds have implicitly filled in all the 
assertions, as was done in Fig. 4. This point of view has important psychological 
consequences for the proper communication of algorithms from one person to 
another: It implies that the key assertions, those that cannot easily be derived 
by an automaton, should always be stated explicitly when an algorithm is being 
explained to someone else. When Algorithm E is being put forward, assertion 
A6 should be mentioned too. 

An alert reader will have noticed a gaping hole in our last proof of Algo- 
rithm E, however. We never showed that the algorithm terminates; all we have 
proved is that if it terminates, it gives the right answer! 

(Notice, for example, that Algorithm E still makes sense if we allow its 
variables m, n, c, d, and r to assume values of the form u + v v2, where u 
and v are integers. The variables q, a, b, a’, b' are to remain integer-valued. 
If we start the method with m = 12 — 6 V2 and n = 20 — 10 V2, say, it will 
compute a “greatest common divisor” d = 4 — 2 V2 with a = +2, b = —1. Even 
under this extension of the assumptions, the proofs of assertions A1 through A6 
remain valid; therefore all assertions are true throughout any execution of the 
procedure. But if we start out with m = 1 and n = V2, the computation never 
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terminates (see exercise 12). Hence a proof of assertions A1 through A6 does 
not logically prove that the algorithm is finite.) 

Proofs of termination are usually handled separately. But exercise 13 shows 
that it is possible to extend the method above in many important cases so that 
a proof of termination is included as a by-product. 

We have now twice proved the validity of Algorithm E. To be strictly logical, 
we should also try to prove that the first algorithm in this section, Algorithm I, 
is valid; in fact, we have used Algorithm I to establish the correctness of any 
proof by induction. If we attempt to prove that Algorithm I works properly, 
however, we are confronted with a dilemma — we can’t really prove it without 
using induction again! The argument would be circular. 

In the last analysis, every property of the integers must be proved using 
induction somewhere along the line, because if we get down to basic concepts, the 
integers are essentially defined by induction. Therefore we may take as axiomatic 
the idea that any positive integer n either equals 1 or can be reached by starting 
with 1 and repetitively adding 1; this suffices to prove that Algorithm I is valid. 
[For a rigorous study of fundamental concepts about the integers, see the article 
“On Mathematical Induction” by Leon Henkin, AMM 67 (1960), 323-338. 

The idea behind mathematical induction is thus intimately related to the 
concept of number. The first European to apply mathematical induction to 
rigorous proofs was the Italian scientist Francesco Maurolico, in 1575. Pierre 
de Fermat made further improvements, in the early 17th century; he called 
it the “method of infinite descent.” The notion also appears clearly in the 
later writings of Blaise Pascal (1653). The phrase “mathematical induction” 
apparently was coined by A. De Morgan in the early nineteenth century. [See 
The Penny Cyclopedia 12 (1838), 465-466; AMM 24 (1917), 199-207; 25 
(1918), 197-201; Arch. Hist. Exact Sci. 9 (1972), 1-21.] Further discussion of 
mathematical induction can be found in G. Pélya’s book Induction and Analogy 
in Mathematics (Princeton, N.J.: Princeton University Press, 1954), Chapter 7. 

The formulation of algorithm-proving in terms of assertions and induction, 
as given above, is essentially due to R. W. Floyd. He pointed out that a semantic 
definition of each operation in a programming language can be formulated as a 
logical rule that tells exactly what assertions can be proved after the operation, 
based on what assertions are true beforehand [see “Assigning Meanings to Pro- 
grams,” Proc. Symp. Appl. Math., Amer. Math. Soc., 19 (1967), 19-32]. Similar 
ideas were voiced independently by Peter Naur, BIT 6 (1966), 310-316, who 
called the assertions “general snapshots.” An important refinement, the notion 
of “invariants,” was introduced by C. A. R. Hoare; see, for example, CACM 14 
(1971), 39-45. Later authors found it advantageous to reverse Floyd’s direction, 
going from an assertion that should hold after an operation to the “weakest 
precondition” that must hold before the operation is done; such an approach 
makes it possible to discover new algorithms that are guaranteed to be correct, 
if we start from the specifications of the desired output and work backwards. 
[See E. W. Dijkstra, CACM 18 (1975), 453-457; A Discipline of Programming 
(Prentice-Hall, 1976).] 
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The concept of inductive assertions actually appeared in embryonic form in 
1946, at the same time as flow charts were introduced by H. H. Goldstine and 
J. von Neumann. Their original flow charts included “assertion boxes” that are 
in close analogy with the assertions in Fig. 4. [See John von Neumann, Collected 
Works 5 (New York: Macmillan, 1963), 91-99. See also A. M. Turing’s early 
comments about verification in Report of a Conference on High Speed Automatic 
Calculating Machines (Cambridge Univ., 1949), 67-68 and figures; reprinted 
with commentary by F. L. Morris and C. B. Jones in Annals of the History of 
Computing 6 (1984), 139-143.] 


The understanding of the theory of a routine 
may be greatly aided by providing, at the time of construction 
one or two statements concerning the state of the machine 
at well chosen points. 
In the extreme form of the theoretical method 
a watertight mathematical proof is provided for the assertions. 
In the extreme form of the experimental method 
the routine is tried out on the machine with a variety of initial 
conditions and is pronounced fit if the assertions hold in each case. 
Both methods have their weaknesses. 


— A. M. TURING, Ferranti Mark | Programming Manual (1950) 


EXERCISES 


1. [05] Explain how to modify the idea of proof by mathematical induction, in case 
we want to prove some statement P(n) for all nonnegative integers — that is, for n = 0, 
1, 2, ... instead of for n = 1, 2, 3,.... 


2. [15] There must be something wrong with the following proof. What is it? 
“Theorem. Let a be any positive number. For all positive integers n we have 


a"! = 1. Proof. If n = 1, a"~' = a'~' = a? = 1. And by induction, assuming 
that the theorem is true for 1, 2, ..., n, we have 
n-1 n—-1 
(n+1)-1__ n_a xa LEXE a 
a a JG- i 1; 


so the theorem is true for n + 1 as well.” 


3. [18] The following proof by induction seems correct, but for some reason the 


equation for n = 6 gives Z H i H b H 5 H a = 2 on the left-hand side, and 3 — 2 = $ 
on the right-hand side. Can you find a mistake? “Theorem. 
t o 1 ' 1 o3 1 
1x2 2x3 (n=1)xn 2 n’ 


Proof. We use induction on n. For n = 1, clearly 3/2 — 1/n = 1/(1 x 2); and, assuming 
that the theorem is true for n, 


1 es 1 1 
1x20 (n=1)xn nx(n+1) 
3 l- d sr Da bs 
2 n n(n+1) 2 n \n ntl 2 n+l 


4. [20] Prove that, in addition to Eq. (3), Fibonacci numbers satisfy F, > ¢°7? 
for all positive integers n. 
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5. [21] A prime number is an integer > 1 that has no positive integer divisors other 
than 1 and itself. Using this definition and mathematical induction, prove that every 
integer > 1 may be written as a product of one or more prime numbers. (A prime 
number is considered to be the “product” of a single prime, namely itself.) 

6. [20] Prove that if Eqs. (6) hold just before step E4, they hold afterwards also. 

7. [23] Formulate and prove by induction a rule for the sums 17,97 17.3297 4 17, 
4? — 37 4.97 — 12 5? — 4? 437 — 97 ete. 

> 8. [25] (a) Prove the following theorem of Nicomachus (A.D. c. 100) by induction: 
18 = 1, 2° = 3+ 5, 33 = 7 +9 +11, 4? = 13 +15 + 17 + 19, etc. (b) Use this result to 
prove the remarkable formula 1° +23 +--- +n? =(14+2+---+n)?. 


[Note: An attractive geometric interpretation of this formula, suggested by Warren 
Lushbaugh, is shown in Fig. 5; see Math. Gazette 49 (1965), 200. The idea is related 
to Nicomachus’s theorem and Fig. 3. Other “look-see” proofs can be found in books 
by Martin Gardner, Knotted Doughnuts (New York: Freeman, 1986), Chapter 16; 
J. H. Conway and R. K. Guy, The Book of Numbers (New York: Copernicus, 1996), 
Chapter 2. 


Side = 5+5+5+5+5+5 =5-(541) 

Side = 5+4+3+2+1+1+2+3+4+5 
=2.(1+2+ --- +5) 

Area = 4-1? +4-2-2? +4-3-3?+4-4-4?+4-5-5° 
= 4.(19 427+... +5°) 


Fig. 5. Geometric version of exercise 8(b). 


9. [20] Prove by induction that if 0 < a < 1, then (1 — a)” > 1 — na. 
10. [M22] Prove by induction that if n > 10, then 2” > n°. 
11. [M30] Find and prove a simple formula for the sum 
il a 5 _ (-1)"(2n +1)? 
1444 34440 


54440" (Qn41)444 ° 


12. [M25] Show how Algorithm E can be generalized as stated in the text so that 
it will accept input values of the form u + vv2, where u and v are integers, and the 
computations can still be done in an elementary way (that is, without using the infinite 
decimal expansion of v2). Prove that the computation will not terminate, however, if 
m = 1 and n = v2. 

> 13. [M23] Extend Algorithm E by adding a new variable T and adding the operation 
“T 4+ T+1” at the beginning of each step. (Thus, T is like a clock, counting the number 
of steps executed.) Assume that T is initially zero, so that assertion A1 in Fig. 4 
becomes “m > 0, n > 0, T = 0.” The additional condition “T = 1” should similarly be 
appended to A2. Show how to append additional conditions to the assertions in such a 
way that any one of A1, A2,..., A6 implies T < 3n, and such that the inductive proof 
can still be carried out. (Hence the computation must terminate in at most 3n steps.) 
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14. [50] (R. W. Floyd.) Prepare a computer program that accepts, as input, programs 
in some programming language together with optional assertions, and that attempts to 
fill in the remaining assertions necessary to make a proof that the computer program 
is valid. (For example, strive to get a program that is able to prove the validity of 
Algorithm E, given only assertions A1, A4, and A6. See the papers by R. W. Floyd 
and J. C. King in the IFIP Congress proceedings, 1971, for further discussion.) 


15. [HM28] (Generalized induction.) The text shows how to prove statements P(n) 
that depend on a single integer n, but it does not describe how to prove statements 
P(m,n) depending on two integers. In these circumstances a proof is often given by 
some sort of “double induction,” which frequently seems confusing. Actually, there 
is an important principle more general than simple induction that applies not only to 
this case but also to situations in which statements are to be proved about uncountable 
sets — for example, P(x) for all real x. This general principle is called well-ordering. 
Let “<” be a relation on a set S, satisfying the following properties: 


i) Given zx, y, and z in S, if x < y and y x z, then x x z. 
ii) Given x and y in S, exactly one of the following three possibilities is true: x < y, 
T=Y, NYT. 
iii) If A is any nonempty subset of S, there is an element x in A with x < y (that is, 
z < yor x = y) forall yin A. 


This relation is said to be a well-ordering of S. For example, it is clear that the positive 
integers are well-ordered by the ordinary “less than” relation, <. 


a) Show that the set of all integers is not well-ordered by <. 

b) 

c) Is the set of all nonnegative real numbers well-ordered by <? 

d) (Lexicographic order.) Let S be well-ordered by <, and for n > 0 let T, be the 
set of all n-tuples (71, £2,..., £n) of elements x; in S. Define (x1, £2,..., En) < 
(y1, Y2,- --, Yn) if there is some k, 1 < k < n, such that x; = yj for 1 < j < k, but 
£k < ye in S. Is < a well-ordering of Tn? 

e) Continuing part (d), let T = Us, Tn; define (#1, 22,...,2m) < (Y y2,---, Yn) if 
zj = yj for 1 < j < k and xy < yx, for some k < min(m,n), or if m < n and 
zj = yj for 1 < j < m. Is < a well-ordering of T? 

f) Show that < is a well-ordering of S if and only if it satisfies (i) and (ii) above and 
there is no infinite sequence x1, £2, £3, ... with £j+1 < x; for all j > 1. 

g) Let S be well-ordered by <, and let P(x) be a statement about the element x of S. 
Show that if P(x) can be proved under the assumption that P(y) is true for all 
y < x, then P(x) is true for all x in S. 


Define a well-ordering relation on the set of all integers. 


[Notes: Part (g) is the generalization of simple induction that was promised; in the case 
S = positive integers, it is just the simple case of mathematical induction treated in the 
text. In that case we are asked to prove that P(1) is true if P(y) is true for all positive 
integers y < 1; this is the same as saying we should prove P(1), since P(y) certainly is 
(vacuously) true for all such y. Consequently, one finds that in many situations P(1) 
need not be proved using a special argument. 

Part (d), in connection with part (g), gives us a powerful method of n-tuple 
induction for proving statements P(m1,..., Mn) about n positive integers mi, ..., Mn. 

Part (f) has further application to computer algorithms: If we can map each state x 
of a computation into an element f(x) belonging to a well-ordered set S, in such a way 
that every step of the computation takes a state x into a state y with f(y) < f(x), then 
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the algorithm must terminate. This principle generalizes the argument about strictly 
decreasing values of n, by which we proved the termination of Algorithm 1.1E.] 


1.2.2. Numbers, Powers, and Logarithms 


Let us now begin our study of numerical mathematics by taking a good look at 
the numbers we are dealing with. The integers are the whole numbers 


Siig —3, —2, —1, 0, 1, 2, 3, ... 


(negative, zero, or positive). A rational number is the ratio (quotient) of two 
integers, p/q, where q is positive. A real number is a quantity x that has a 
decimal expansion 


x = n + 0.didəd3 ..., (1) 


where n is an integer, each d; is a digit between 0 and 9, and the sequence of 
digits doesn’t end with infinitely many 9s. The representation (1) means that 
dy d2 dk dı də dk 1 
t t oer et < t t Pee t ‘ 
i 10k SPS” E To wte O 


for all positive integers k. Examples of real numbers that are not rational are 


T = 3.14159265358979 ..., the ratio of circumference to diameter in a circle; 
$ = 1.61803398874989 ..., the golden ratio (1 + v5 )/2 (see Section 1.2.8). 


A table of important constants, to forty decimal places of accuracy, appears in 
Appendix A. We need not discuss the familiar properties of addition, subtrac- 
tion, multiplication, division, and comparison of real numbers. 

Difficult problems about integers are often solved by working with real 
numbers, and difficult problems about real numbers are often solved by working 
with a still more general class of values called complex numbers. A complex 
number is a quantity z of the form z = x + iy, where x and y are real and 7 is 
a special quantity that satisfies the equation i? = —1. We call x and y the real 
part and imaginary part of z, and we define the absolute value of z to be 


|z| = Va? +y. (3) 


The complex conjugate of z is Z = x — iy, and we have zZ = x? +y? = |z|?. The 
theory of complex numbers is in many ways simpler and more beautiful than 
the theory of real numbers, but it is usually considered to be an advanced topic. 
Therefore we shall concentrate on real numbers in this book, except when real 
numbers turn out to be unnecessarily complicated. 

If u and v are real numbers with u < v, the closed interval [u ..v] is the set 
of real numbers g such that u < x < v. The open interval (u..v) is, similarly, 
the set of x such that u <a < v. And half-open intervals [w..v) or (u..v] are 
defined in an analogous way. We also allow u to be —oo or v to be œ at an 
open endpoint, meaning that there is no lower or upper bound; thus (—oo .. 00) 
stands for the set of all real numbers, and [0..00) denotes the nonnegative reals. 
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Throughout this section, let the letter b stand for a positive real number. If 
n is an integer, then b” is defined by the familiar rules 


b° = 1, b” =o" if n>0, Bap Ys if n= 0. (4) 
It is easy to prove by induction that the laws of exponents are valid: 
bere = PW, (b")¥ = bY, (5) 


whenever x and y are integers. 

If u is a positive real number and if m is a positive integer, there is always 
a unique positive real number v such that v™ = u; it is called the mth root of u, 
and denoted v = Yu. 

We now define b” for rational numbers r = p/q as follows: 


bPa — Vp. (6) 


This definition, due to Oresme (c. 1360), is a good one, since b%?/¢4 = b?/4, and 
since the laws of exponents are still correct even when x and y are arbitrary 
rational numbers (see exercise 9). 

Finally, we define b” for all real values of x. Suppose first that b > 1; if x is 
given by Eq. (1), we want 


k k k 
prtds/10+--+de/10" < pe < pntdi/10+--+dx,/10"+1/10" (7) 


This defines b” as a unique positive real number, since the difference between the 
right and left extremes in Eq. (7) is O° +42/10+--+de/10* (61/10" _ 1). by exercise 13 
below, this difference is less than 6"+!(b—1)/10*, and if we take k large enough, 
we can therefore get any desired accuracy for b”. 

For example, we find that 


100-30102999 — 1.9999999739..., 100-50108000 = 2.9000000199...; (8) 


therefore if b = 10 and x = 0.30102999..., we know the value of b” with an 
accuracy of better than one part in 10 million (although we still don’t even know 
whether the decimal expansion of b” is 1.999... or 2.000...). 

When b < 1, we define b” = (1/b)~*; and when b = 1, b” = 1. With 
these definitions, it can be proved that the laws of exponents (5) hold for any 
real values of x and y. These ideas for defining b” were first formulated by John 
Wallis (1655) and Isaac Newton (1669). 

Now we come to an important question. Suppose that a positive real number 
y is given; can we find a real number x such that y = b”? The answer is “yes” 
(provided that b 4 1), for we simply use Eq. (7) in reverse to determine n and 


di, d2,... when b” = y is given. The resulting number zx is called the logarithm 
of y to the base b, and we write this as x = log, y. By this definition we have 
a = bV% = log, (b). (9) 


As an example, Eqs. (8) show that 
logio 2 = 0.30102999... . (10) 
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From the laws of exponents it follows that 
log, (zy) = log, x + logy, if «>0, y>0 (11) 


and 
log,(c’) = y logc, if c>0. (12) 


Equation (10) illustrates the so-called common logarithms, which we get 
when the base is 10. One might expect that in computer work binary logarithms 
(to the base 2) would be more useful, since most computers do binary arithmetic. 
Actually, we will see that binary logarithms are indeed very useful, but not only 
for that reason; the reason is primarily that a computer algorithm often makes 
two-way branches. Binary logarithms arise so frequently, it is wise to have a 
shorter notation for them. Therefore we shall write 


lg x = logs x, (13) 


following a suggestion of Edward M. Reingold. 
The question now arises as to whether or not there is any relationship 
between lg x and log, x; fortunately there is, 


logio £ = log10(28®) = (lg x) (logy, 2), 
by Eqs. (9) and (12). Hence lg x = logo x/log4o 2, and in general we find that 


log, x 
l = : 
aa aget (14) 


Equations (11), (12), and (14) are the fundamental rules for manipulating log- 
arithms. 

It turns out that neither base 10 nor base 2 is really the most conve- 
nient base to work with in most cases. There is a real number, denoted by 
e = 2.718281828459045..., for which the logarithms have simpler properties. 
Logarithms to the base e are conventionally called natural logarithms, and we 
write 

Ina = log, x. (15) 
This rather arbitrary definition (in fact, we haven’t really defined e) probably 
doesn’t strike the reader as being a very “natural” logarithm; yet we’ll find that 
In z seems more and more natural, the more we work 
with it. John Napier actually discovered natural 
logarithms (with slight modifications, and without 
connecting them with powers) before the year 1590, 
many years before any other kind of logarithm was 


known. The following two examples, proved in ev- on 

ery calculus text, shed some light on why Napier’s (x, 1/x) 
logarithms deserve to be called “natural”: (a) In 

Fig. 6 the area of the shaded portion is Ina. (b) Ifa (1,0) (x, 0) 


bank pays compound interest at rate r, compounded 


Fig. 6. Natural logarithm. 
semiannually, the annual return on each dollar is 
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(1+ r/2)? dollars; if it is compounded quarterly, you get (1 + r/4)* dollars; 
and if it is compounded daily you probably get (1 + 7/365)? dollars. Now if 
the interest were compounded continuously, you would get exactly e” dollars for 
every dollar (ignoring roundoff error). In this age of computers, many bankers 
have now actually reached the limiting formula. 

The interesting history of the concepts of logarithm and exponential has 
been told in a series of articles by F. Cajori, AMM 20 (1913), 5-14, 35-47, 
75-84, 107-117, 148-151, 173-182, 205-210. 

We conclude this section by considering how to compute logarithms. One 
method is suggested immediately by Eq. (7): If we let b” = y and raise all parts 
of that equation to the 10*-th power, we find that 


p™ < yo" < pert. (16) 


for some integer m. All we have to do to get the logarithm of y is to raise y to 
this huge power and find which powers (m, m+ 1) of b the result lies between; 
then m/10* is the answer to k decimal places. 

A slight modification of this apparently impractical method leads to a simple 
and reasonable procedure. We will show how to calculate logo x and to express 
the answer in the binary system, as 


logio £ = n+ by /2 + b2/4 + b3/8 +--+. (17) 


First we shift the decimal point of x to the left or to the right so that we have 
1 < 2/10" < 10; this determines the integer part, n. To obtain bi, b2, ..., we 
now set % = x/10” and, for k > 1, 


bk =0, te= L? 43 if TTAN < 10; (18) 
bk=1, Tk= x?_;/10, if A > 10. 
The validity of this procedure follows from the fact that 
1 < ap = 2/19 0401/240/2") < 10, (19) 


for k =0,1,2,..., as is easily proved by induction. 

In practice, of course, we must work with only finite accuracy, so we cannot 
set £k = r7_, exactly. Instead, we set xy = 27_, rounded or truncated to a 
certain number of decimal places. For example, here is the evaluation of log, 2 
rounded to four significant figures: 


xo = 2.000; 

xı = 4.000, bı 
T2 = 1.600, bo = 
v3 = 2.560, b3 = 
t4 = 6.554, b4 = 
zs = 4.295, bs = 


7 


BS Sa SS 


rE = 1.845, be = 1 
3.404, br = 0 

zg = 1.159, bẹ = 1: 
0 

0 


8 
3 
II 


gtg = 1.343, bg = 
; zio = 1.804, bio = 


; etc. 


Computational error has caused errors to propagate; the true rounded value of 
z19 is 1.798. This will eventually cause big to be computed incorrectly, and 
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we get the binary value (0.0100110100010000011...)2, which corresponds to the 
decimal equivalent 0.301031... rather than the true value given in Eq. (10). 

With any method such as this it is necessary to examine the amount of 
computational error due to the limitations imposed. Exercise 27 derives an 
upper bound for the error; working to four figures as above, we find that the 
error in the value of the logarithm is guaranteed to be less than 0.00044. Our 
answer above was more accurate than this primarily because ro, £1, £2, and x3 
were obtained exactly. 

This method is simple and quite interesting, but it is probably not the 
best way to calculate logarithms on a computer. Another method is given in 
exercise 25. 


EXERCISES 
1. [00] What is the smallest positive rational number? 
2. [00] Is 1+ 0.239999999... a decimal expansion? 
3. [02] What is (—3)~?? 
> 4. [05] What is (0.125)~?/3? 


5. [05] We defined real numbers in terms of a decimal expansion. Discuss how we 
could have defined them in terms of a binary expansion instead, and give a definition 
to replace Eq. (2). 


6. [10] Let x = m + 0.dıd2 ... and y = n + 0.e1e2 ... be real numbers. Give a rule 
for determining whether x = y, x < y, or x > y, based on the decimal representation. 

7. [M23] Given that x and y are integers, prove the laws of exponents, starting from 
the definition given by Eq. (4). 

8. [25] Let m be a positive integer. Prove that every positive real number u has a 
unique positive mth root, by giving a method to construct successively the values n, 
di, d2, ... in the decimal expansion of the root. 

9. [M23] Given that x and y are rational, prove the laws of exponents under the 
assumption that the laws hold when zx and y are integers. 


10. [18] Prove that log,, 2 is not a rational number. 


> 11. [10] If b= 10 and x & logy, 2, to how many decimal places of accuracy will we 
need to know the value of x in order to determine the first three decimal places of 
the decimal expansion of b”? [Note: You may use the result of exercise 10 in your 
discussion. ] 


12. [02] Explain why Eq. (10) follows from Eqs. (8). 


> 13. [M23] (a) Given that x is a positive real number and n is a positive integer, prove 
the inequality V1 +x£x—1 < z/n. (b) Use this fact to justify the remarks following (7). 


14. [15] Prove Eq. (12). 
15. [10] Prove or disprove: 
log, z/y = log, x —log,y, if x,y >0. 
16. [00| How can log, x be expressed in terms of ln x and In 10? 
> 17. [05] What is lg32? log, a? Ine? log, 1? log,(—1)? 
18. [10] Prove or disprove: logg x = 4 lg x. 
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> 19. [20] Ifn is an integer whose decimal representation is 14 digits long, will the value 
of n fit in a computer word with a capacity of 47 bits and a sign bit? 


20. [10] Is there any simple relation between log), 2 and log, 10? 
21. [15] (Logs of logs.) Express log, log, x in terms of InInz, InInb, and Inb. 
> 22. [20] (R. W. Hamming.) Prove that 


lgx ~ lng + logio @, 


with less than 1% error! (Thus a table of natural logarithms and of common logarithms 
can be used to get approximate values of binary logarithms as well.) 


23. [M25] Give a geometric proof that ln xy = ln z + ln y, based on Fig. 6. 


24. [15] Explain how the method used for calculating logarithms to the base 10 at 
the end of this section can be modified to produce logarithms to base 2. 


25. [22] Suppose that we have a binary computer and a number z, 1 < x < 2. 
Show that the following algorithm, which uses only shifting, addition, and subtraction 
operations proportional to the number of places of accuracy desired, may be used to 
calculate an approximation to y = log, x: 


L1. [Initialize.] Set y 4+ 0, z + x shifted right 1, k + 1. 

L2. [Test for end.] If x = 1, stop. 

L3. [Compare.] If x — z < 1, set z + z shifted right 1, k + k +1, and repeat this 
step. 

L4. [Reduce values.] Set z + x—z, z + z shifted right k, y + y+log,(2*/(2*—1)), 
and go to L2. I 


[Notes: This method is very similar to the method used for division in computer 
hardware. The idea goes back in essence to Henry Briggs, who used it (in decimal 
rather than binary form) to compute logarithm tables, published in 1624. We need 
an auxiliary table of the constants log, 2, log,(4/3), log,(8/7), etc., to as many values 
as the precision of the computer. The algorithm involves intentional computational 
errors, as numbers are shifted to the right, so that eventually x will be reduced to 1 
and the algorithm will terminate. The purpose of this exercise is to explain why it will 
terminate and why it computes an approximation to log, x. ] 


26. [M27] Find a rigorous upper bound on the error made by the algorithm in the 
previous exercise, based on the precision used in the arithmetic operations. 


> 27. [M25] Consider the method for calculating log,, x discussed in the text. Let £p 
denote the computed approximation to xg, determined as follows: x(1—6) < 10”xb < 
x(1+ €); and in the determination of x}, by Eqs. (18), the quantity yx is used in place 
of (a@,_1)?, where (x11) (1 — 5) < ye < (x,_1)?(1 + ©) and 1 < yk < 100. Here 6 
and e are small constants that reflect the upper and lower errors due to rounding or 
truncation. If log’ x denotes the result of the calculations, show that after k steps we 
have 

logio £ + 2log,)(1 — 5) — 1/2" < log’ x < logio £ + 2log;9(1 + ©). 

28. [M30] (R. Feynman.) Develop a method for computing b” when 0 < x < 1, using 
only shifting, addition, and subtraction (similar to the algorithm in exercise 25), and 
analyze its accuracy. 
29. [HM20] Let x bea real number greater than 1. (a) For what real number b > 1 is 
blog, x a minimum? (b) For what integer b > 1 is it a minimum? (c) For what integer 
b > 1 is (b + 1) log, z a minimum? 
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30. [12] Simplify the expression (Inz)"™*/""*, assuming that x > 1 and x £ e. 


1.2.3. Sums and Products 

Let a), a@2,... be any sequence of numbers. We are often interested in sums such 
as a1 +d2+---+a,, and this sum is more compactly written using either of the 
following equivalent notations: 


n 
Xaj or 5 Qj. (1) 
j=l I<j<n 
If n is zero, the value of aj + a2 +--+ an = a9 = vi<j<n%i is defined to 
be zero. Our convention of using “three dots” in sums such as aj + a2 +--+ an 
therefore has some slightly peculiar, but sensible, behavior in borderline cases 
(see exercise 1). 
In general, if R(j) is any relation involving 7, the symbol 


Xa (2) 

RG) 
means the sum of all a; where j is an integer satisfying the condition R(j). If 
no such integers exist, notation (2) denotes zero. The letter j in (1) and (2) 
is a dummy index or index variable, introduced just for the purposes of the 
notation. Symbols used as index variables are usually the letters i, j, k, m, n, r, 
s, t (occasionally with subscripts or accent marks). Large summation signs like 
those in (1) and (2) can also be rendered more compactly as ae aj Or DRG) Qj. 
The use of a ` and index variables to indicate summation with definite limits 
was introduced by J. Fourier in 1820. 

Strictly speaking, the notation )7,- j<n 4j ÍS ambiguous, since it does not 
clarify whether the summation is taken with respect to j or to n. In this 
particular case it would be rather silly to interpret it as a sum on values of n > j; 
but meaningful examples can be constructed in which the index variable is not 
clearly specified, as in ` j<k (25: In such cases the context must make clear 
which variable is a dummy variable and which variable has a significance that 
extends beyond its appearance in the sum. A sum such as Ð j< (2 would 
presumably be used only if either j or k (not both) has exterior significance. 

In most cases we will use notation (2) only when the sum is finite — that is, 
when only a finite number of values j satisfy R(j) and have a; # 0. If an infinite 
sum is required, for example 


co 
Soa; = Xaj = a, +a2 +03 +: 
j=1 


J21 
with infinitely many nonzero terms, the techniques of calculus must be employed; 
the precise meaning of (2) is then 


Ya = (tim, D as) + (tim a), () 
RO) Eo PD) 
Sg<n =P] 
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provided that both limits exist. If either limit fails to exist, the infinite sum is 
divergent; it does not exist. Otherwise it is convergent. 

When two or more conditions are placed under the X` sign, as in (3), we 
mean that all conditions must hold. 


Four simple algebraic operations on sums are very important, and familiarity 
with them makes the solution of many problems possible. We shall now discuss 
these four operations. 


a) The distributive law, for products of sums: 


EN(E)-EE4) v 


R(i) Ri) *S(j) 


To understand this law, consider for example the special case 


(>: ai) (È ») = (aı + a2)(bı + b2 + b3) 


= (aibı t aıb2 } a1b3) t (a2by t agbo t abs) 


It is customary to drop the parentheses on the right-hand side of (4); a double 
summation such as >? pa) ( so) aij) is written simply Lro so) tij 
b) Change of variable: 


a=) y= SP ao (5) 
R(j) 


R(i) R(p(3)) 


This equation represents two kinds of transformations. In the first case we are 
simply changing the name of the index variable from i to j. The second case 
is more interesting: Here p(j) is a function of j that represents a permutation 
of the relevant values; more precisely, for each integer i satisfying the relation 
R(t), there must be exactly one integer j satisfying the relation p(j) = i. This 
condition is always satisfied in the important cases p(j) = c+ j and p(j) = c—J, 
where c is an integer not depending on j, and these are the cases used most 
frequently in applications. For example, 


5 aâj = 5 aj-1 = 5 aj-1.- (6) 


1<j<n 1<j-l<n 2<j<n+1 


The reader should study this example carefully. 

The replacement of j by p(j) cannot be done for all infinite sums. The 
operation is always valid if p(j) = c j, as above, but in other cases some 
care must be used. [For example, see T. M. Apostol, Mathematical Analysis 
(Reading, Mass.: Addison-Wesley, 1957), Chapter 12. A sufficient condition to 
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guarantee the validity of (5) for any permutation of the integers, p(j), is that 
Vr) lal exists. 
c) Interchanging order of summation: 
> 205 = 2 dais: (7) 
R(i) S S(j) RÇ) 


Let us consider a very simple special case of this equation: 


5 5 Qij = 5 (aii + ai2), 


R(i) j=1 R(i) 

2 
5 5 iya ue 
j=1 RÇ) R(i) 


By Eq. (7), these two are equal; this says no more than 
Lata Dat Da o) 
R(i) 


where we let b; = a;; and c; = ais. 

The operation of interchanging the order of summation is extremely useful, 
since it often happens that we know a simple form for >> R(i) tij» but not for 
5 S(j) tij: We frequently need to interchange the summation order also in a 
more general situation, where the relation S(j) depends on i as well as 7. In such 
a case we can denote the relation by “S(i,7).” The interchange of summation 
can always be carried out, in theory at least, as follows: 


dy ead, Dee (9) 


R(t) S(i,j) S' (j) R' (i,j) 


where $’(j) is the relation “there is an integer i such that both R(i) and S(i, 7) 
are true”; and R’(i,j) is the relation “both R(i) and S(i,j) are true.” For 
example, if the summation is }7j_, )°4—1 ij, then S$‘(j) is the relation “there is 
an integer i such that 1 < i < n and 1 < j <i,” that is, 1 < j < n; and R'(i,7) 
is the relation “1 < i < n and 1 < j < i,” that is, j < i < n. Thus, 


3 Ya => Dew. Go) 


[Note: As in case (b), the operation of interchanging order of summation is not 
always valid for infinite series. If the series is absolutely convergent — that is, if 
X ra 2o sg) |@ij| exists —it can be shown that Eqs. (7) and (9) are valid. Also 
if either one of R(i) or S(j) specifies a finite sum in Eq. (7), and if each infinite 
sum that appears is convergent, then the interchange is justified. In particular, 
Eq. (8) is always true for convergent infinite sums. 
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d) Manipulating the domain. If R(j) and S(j) are arbitrary relations, we 


have 
So as+ > a; = 5 aj + 5 Qj. (11) 


RG) 5(3) R(Jj) or S(9) R(j) and S(j) 
For example, 
5 aj + 5 aj = ( 5 ai) +am, (12) 
l<jgm m<jen 1<j<n 


assuming that 1 < m < n. In this case “R(j) and S(j)” is simply “j = m,” so we 
have reduced the second sum to simply “am.” In most applications of Eq. (11), 
either R(j) and S(j) are simultaneously satisfied for only one or two values of j, 
or else it is impossible to have both R(j) and S(j) true for the same j. In the 
latter case, the second sum on the right-hand side of Eq. (11) simply disappears. 

Now that we have seen the four basic rules for manipulating sums, let’s 
study some further illustrations of how to apply these techniques. 


Example 1. 
X aj = > aj + > aj by rule (d) 
0<j<n 0<j<n 0<j<n 
j even jodd 
= ) a2j + J a2j+1 by rule (b) 
O<2j<n O<2j+1<n 
2j even 2j+1 odd 
= X a2j + X Q2j+1- 
O<j<n/2 0<j<n/2 


The last step merely consists of simplifying the relations below the }°’s. 
Example 2. Let 


Ss, = 5 X aia; = 5 X aia, by rule (c) [see Eq. (10)] 


i=0 j=0 j=0 i=j 


= 5 > aiaj by rule (b), 


i=0 j=i 


interchanging the names 7 and j and recognizing that aja; = ajaj. If we denote 
the latter sum by S2, we have 


n 


2S) = Si + So= 5 > aja; + 5 aa) by Eq. (8) 
j=0 j=i 


i=0 


£ Z by rule (d 
= D (2 aias ) + na) > ia bai 
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by Eq. (8) 


= ( ai) 62 as) + ( a?) by rule (a) 
i=0 j=0 i=0 


_ ( n a) ie (>: a?) by rule (b). 
i=0 i=0 


Thus we have derived the important identity 


Stet (Se) (Ee) 


i=0 j=0 i=0 i=0 


II 

S 
Q 
S 

+ 
a 
S 

£ 


>N 


Example 3 (The sum of a geometric progression). Assume that x # 1 and that 
n > 0. Then 


a+ar+---+ar" = 5 ari by definition (2) 
OSj<n 
=a + 5 ax! by rule (d) 
1<j<n 

=a+r 5 ax)! by a very special case of (a) 
1<j<n 

=a+ez 5 axi by rule (b) [see Eq. (6)] 
0<j<n-1 

=a+2 ` ax —ax"t! by rule (d). 


OSj<n 
Comparing the first relation with the last, we have 
(1-2) 5 azi = a — ax”tt, 
OSj<n 
hence we obtain the basic formula 
. 1— n+1 
5 ax! = o(—_). (14) 
- l-z 
OSj<n 
Example 4 (The sum of an arithmetic progression). Assume that n > 0. Then 
a+ (a+b) +--+ (a + nb) 
= 5 (a+ bj) by definition (2) 


O<j<n 
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= 5 (a + b(n — j)) by rule (b) 
O0<n—j<n 

= 5 (a + bn — bj) by simplification 
O<jxn 

= X @a+bn)- X` (a+bj) by Eq. (8) 
O<j<n O<jxn 

=(n+1)(2a+bn)— X` (a+b), 

O<jen 


since the first sum simply adds together (n + 1) terms that do not depend on j. 
Now by equating the first and last expressions and dividing by 2, we obtain 


5 (a+bj)=a(n +1) + $bn(n +1). (15) 
O<jegn 


This is n + 1 times $(a+ (a + bn)), which can be understood as the number of 


terms times the average of the first and last terms. 

Notice that we have derived the important equations (13), (14), and (15) 
purely by using simple manipulations of sums. Most textbooks would simply 
state those formulas, and prove them by induction. Induction is, of course, a 
perfectly valid procedure; but it does not give any insight into how on earth 
a person would ever have dreamed the formula up in the first place, except by 
some lucky guess. In the analysis of algorithms we are confronted with hundreds 
of sums that do not conform to any apparent pattern; by manipulating those 
sums, as above, we can often get the answer without the need for ingenious 
guesses. 

Many manipulations of sums and other formulas become considerably sim- 
pler if we adopt the following bracket notation: 


1, if the statement is true; 
(16) 


ptatem] = { 0, if the statement is false. 


Then we can write, for example, 


X a= > a [RO], (17) 
R(i) 


J 


where the sum on the right is over all integers 7, because the terms of that 
infinite sum are zero when R(j) is false. (We assume that a; is defined for all j.) 

With bracket notation we can derive rule (b) from rules (a) and (c) in an 
interesting way: 


X ayy = > ayy [R@W))] 
R(p(3)) j 
= YS a: [RO] [t=] 
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= da [RO] J [i= rd]. (18) 


J 


The remaining sum on j is equal to 1 when R(i) is true, if we assume that p is 
a permutation of the relevant values as required in (5); hence we are left with 
>, ai[R(4)], which is > Ro) a;. This proves (5). If p is not such a permutation, 
(18) tells us the true value of >? rR) p0) 

The most famous special case of bracket notation is the so-called Kronecker 
delta symbol, 


o. l, ifi=j, 
yaksi Bind (19) 


introduced by Leopold Kronecker in 1868. More general notations such as (16) 
were introduced by K. E. Iverson in 1962; therefore (16) is often called Iverson’s 
convention. [See D. E. Knuth, AMM 99 (1992), 403-422.] 


There is a notation for products, analogous to our notation for sums: The 


symbols 
II Qj (20) 
R(j) 
stand for the product of all a; for which the integer j satisfies R(j). If no such 
integer j exists, the product is defined to have the value 1 (not 0). 

Operations (b), (c), and (d) are valid for the []-notation as well as for the 
Š -notation, with suitable simple modifications. The exercises at the end of this 
section give a number of examples of product notation in use. 

We conclude this section by mentioning another notation for multiple sum- 
mation that is often convenient: A single -sign may be used with one or 
more relations in several index variables, meaning that the sum is taken over all 
combinations of variables that meet the conditions. For example, 


X X Qij = X Qij; X X Qij = X Qij. 
O<i<n O<j<n O<i,j<n O<i<n O<j<i O<j<i<n 


This notation gives no preference to one index of summation over any other, so 
it allows us to derive (10) in a new way: 


Y J aig = Sais [1 sicn <j <i] = Yo a1 <j <n]lj<i<nl 


i=1 j=1 ij ij 
=), da, 
j=l i=j 
using the fact that [1<i<n][1 <j <i] =[1<j<i<n]=[1<j<n]lj<i<nl. 


The more general equation (9) follows in a similar way from the identity 


[RO] [SGI] = [R@ and S4, 5)] = [S0] [RG] (21) 
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A further example that demonstrates the usefulness of summation with 


several indices is 
X ) Qji.jn? (22) 


jit +in=n 
fiz" jn20 


where a is an n-tuply subscripted variable; for example, if n = 5 this notation 
stands for 


@11111 + 421110 + @22100 + 431100 + 432000 + @41000 + 450000. 


(See the remarks on partitions of a number in Section 1.2.1.) 


EXERCISES — First Set 
1. [10] The text says that a1 + a2 + --- + ao = 0. What, then, is a2 +---+ ao? 
2. [01] What does the notation >), <;<, 4j mean, if n = 3.14? 
3. [13] Without using the }>-notation, write out the equivalent of 
1 


and also the equivalent of 


Explain why the two results are different, in spite of rule (b). 


4. [10] Without using the }>-notation, write out the equivalent of each side of 
Eq. (10) as a sum of sums for the case n = 3. 


5. [HM20] Prove that rule (a) is valid for arbitrary infinite series, provided that the 
series converge. 


6. [HM20] Prove that rule (d) is valid for an arbitrary infinite series, provided that 
any three of the four sums exist. 


7. [HM23] Given that c is an integer, show that ’` ro) aj = J rej) Ges, even if 
both series are infinite. 


8. [HM25] Find an example of infinite series in which Eq. (7) is false. 
9. [05] Is the derivation of Eq. (14) valid even if n = —1? 

10. [05] Is the derivation of Eq. (14) valid even if n = —2? 

11. [03] What should the right-hand side of Eq. (14) be if x = 1? 


12. [10] What is 1+ $+ + ṣg +o +(+)? 


13. [10] Using Eq. (15) and assuming that m < n, evaluate >; m J. 


14. [11] Using the result of the previous exercise, evaluate a nL 


15. [M22] Compute the sum 1x 2+2x2?+43x2°+---+nx2” for small values of n. Do 
you see the pattern developing in these numbers? If not, discover it by manipulations 
similar to those leading up to Eq. (14). 


v 


v 


v 
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16. [M22] Prove that 


S — nat? —(n+ lett +a 
2 CEE 


if x £ 1, without using mathematical induction. 
17. [M00] Let S be a set of integers. What is -jes 1? 


18. [M20] Show how to interchange the order of summation as in Eq. (9) given that 
R(t) is the relation “n is a multiple of i” and S(i, j) is the relation “1 < j < i.” 


19. [20] What is pe C7, = aj—1)? 


20. [25] Dr. I. J. Matrix has observed a remarkable sequence of formulas: 


9x142=11, 9x 124+3=111,9x 123 + 4 = 1111, 9 x 1234 + 5 = 11111. 


a) Write the good doctor’s great discovery in terms of the `-notation. 


b) Your answer to part (a) undoubtedly involves the number 10 as base of the decimal 
system; generalize this formula so that you get a formula that will perhaps work 
in any base b. 


c) Prove your formula from part (b) by using formulas derived in the text or in 
exercise 16 above. 


21. [M25] Derive rule (d) from (8) and (17). 


22. [20] State the appropriate analogs of Eqs. (5), (7), (8), and (11) for products 
instead of sums. 


23. [10] Explain why it is a good idea to define `p) aj and Į [r)a as zero and 
one, respectively, when no integers satisfy R(j). 


24. [20] Suppose that R(j) is true for only finitely many j. By induction on the 
number of integers satisfying R(j), prove that log, Į [rg 47 = X rg) (log, aj), assuming 
that all a; > 0. 


25. [15] Consider the following derivation; is anything amiss? 


(a)(EL)-E ESE Es bi 


i=l j=l l<i<n 1l<j<n 1l<i<n 1<i<n 


26. [25] Show that []?_, M aiaj may be expressed in terms of ]]?_, a; by manip- 
ulating the []-notation as stated in exercise 22. 


27. [M20] Generalize the result of exercise 1.2.1-9 by proving that 


n 


assuming that 0 < a; < 1. 
28. [M22] Find a simple formula for [[_, (1 — 1/37). 
29. [M30] (a) Express }>"_, Y= Slo @iajax in terms of the multiple-sum notation 


explained at the end of the section. (b) Express the same sum in terms of )>).9 ai, 
7 5 @;, and >," oa; [see Eq. (13)]. 
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> 30. [M23] (J. Binet, 1812.) Without using induction, prove the identity 


> aa) (£ bu) = (>: an) 2 bias) + J (ajbk—akbj)(£jYyk—z£ryi). 


1<j<k<n 


[An important special case arises when wi, ..., Wn, 21, ---; Zn are arbitrary complex 
numbers and we set aj = wj, bj = Zj, £j = Wj, Yj = Zj: 


(Ze) (Ziar) 2 Swe 


The terms |wjZk — wkZj|? are nonnegative, so the famous Cauchy—Schwarz inequality 


(Zee) (Zr) > Duz 


is a consequence of Binet’s formula.] 


2 
+ 5 [WwjZk — wpZ;|*. 


1<j<k<n 


2 


31. [M20] Use Binet’s formula to express the sum $` ;<j<k<n(Uj — Uk)(Vj — Ve) in 
terms of J 5—1 ujvj, J j= us, and } o}; vj. an 


32. [M20] Prove that 


| | Qij = > Qiz1-+-Qinn- 


j=l i=1 l<ig, in lm 


> 33. [M30] One evening Dr. Matrix discovered some formulas that might even be 
classed as more remarkable than those of exercise 20: 


1 1 1 


a—b)(a—c) b—a)(b—c) Seay Y 
a b c -o 
a—b)(a—c) (b-—a)(b—c) (c—a)(c—b) , 
a? b? e th 
a—b)(a—c) b—a)(b—c) c—a)(c—b) i 
a? b? 3 
=a+b+c. 
a—b)(a—c) b—a)(b—c) c—a)(c—b) 
Prove that these formulas are a special case of a general law; let £1, £2,..., £n be 
distinct numbers, and show that 
n 0, if0<r<n-l1,; 
S(s ia (e) -m) )= L ae 
j=l 1<k<n Va tj ifr=n. 


Aj 
34. [M25] Prove that 


2 Thies. rm (x +k- r) 


3 =1 


k=1 Ther<n, ren (® =r) 
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provided that 1 < m < n and z is arbitrary. For example, if n = 4 and m = 2, then 
g(x — 2)(x—3) (x+1)(x-—1)(x-— 2) pi (x +2)x(x—-1) (£+3)(£+1)z 
(—1)(—2)(-3) (1)(—1)(—2) DDED (3)(2)(4) 


35. [HM20] The notation sup,,;) aj is used to denote the least upper bound of the 
elements aj, in a manner exactly analogous to the }>- and [[-notations. (When R(j) 
is satisfied for only finitely many j, the notation max g(;) a; is often used to denote the 
same quantity.) Show how rules (a), (b), (c), and (d) can be adapted for manipulation 
of this notation. In particular discuss the following analog of rule (a): 


=I. 


(SUP R(:) ai) + (sup 5(;) bj) = SUP gç) (SUP 5 (5) (Gi + b;)), 


and give a suitable definition for the notation when R(j) is satisfied for no j. 


EXERCISES — Second Set 


Determinants and matrices. The following interesting problems are for the reader 
who has experienced at least an introduction to determinants and elementary matrix 
theory. A determinant may be evaluated by astutely combining the operations of: (a) 
factoring a quantity out of a row or column; (b) adding a multiple of one row (or 
column) to another row (or column); (c) expanding by cofactors. The simplest and 
most often used version of operation (c) is to simply delete the entire first row and 
column, provided that the element in the upper left corner is +1 and the remaining 
elements in either the entire first row or the entire first column are zero; then evaluate 
the resulting smaller determinant. In general, the cofactor of an element a;; in an 
n x n determinant is (—1)'*? times the (n — 1) x (n — 1) determinant obtained by 
deleting the row and column in which aj; appeared. The value of a determinant is 
equal to ` aij - cofactor(a;;) summed with either i or j held constant and with the 
other subscript varying from 1 to n. 

If (bij) is the inverse of matrix (aij), then bj; equals the cofactor of aji (not aij), 
divided by the determinant of the whole matrix. 

The following types of matrices are of special importance: 


Vandermonde’s matriz, Combinatorial matriz, 
Qij = z Qij = Y + Õij£ 
Tı LQ... Tn xr+YyY y reg Y 
r? x2 sited r2 y THY es. Y 
at ey... Tr y y .-.- THY 


Cauchy’s matrix, 

aij = 1/(x: + ys) 
1/(z1+y1) 1/(z1ı +y2) ... 1/(t1 + yn) 
1/(z2+y1ı) 1/(z2+y2) ... 1/(£2 + yn) 


1/(£n +y1) 1/(@n+y2) ... 1/(£n + yn) 
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36. [M23] Show that the determinant of the combinatorial matrix is x"~1(x + ny). 
37. [M24] Show that the determinant of Vandermonde’s matrix is 
H = TI ee 
1<j<n  1si<jsn 


38. [M25] Show that the determinant of Cauchy’s matrix is 


I] -zu -) [[ @i+y). 


1<i<j<n 1<i,j<n 
39. [M23] Show that the inverse of a combinatorial matrix is a combinatorial matrix 
with the entries bij = (—y + ôi; (x + ny))/a(x+ ny). 
40. [M24] Show that the inverse of Vandermonde’s matrix is given by 


bij = 5 (=1) tig se egy Li II e Lk — Xi) 


1<k1 <- <kn—j<n 1<k<n 
k1,..kn—7t kAt 


Don’t be dismayed by the complicated sum in the numerator — it is just the coefficient 
of 23~' in the polynomial (xı — x)... (an — x)/(ai — x). 


41. [M26] Show that the inverse of Cauchy’s matrix is given by 


bij = ( I] Gi + yx)(ex +w)) [te +wo( J[ œ -2))( IL w -m)): 


1<k<n 1<k<n 1<k<n 
k£j kżi 


42. [M18] What is the sum of all n? elements in the inverse of the combinatorial 
matrix? 


43. [M24] What is the sum of all n? elements in the inverse of Vandermonde’s matrix? 
[Hint: Use exercise 33.] 


44. [M26] What is the sum of all n? elements in the inverse of Cauchy’s matrix? 


45. [M25] A Hilbert matrix, sometimes called an nxn segment of the (infinite) Hilbert 
matrix, is a matrix for which a;; = 1/(i + j — 1). Show that this is a special case of 
Cauchy’s matrix, find its inverse, show that each element of the inverse is an integer, 
and show that the sum of all elements of the inverse is n?. [Note: Hilbert matrices 
have often been used to test various matrix manipulation algorithms, because they 
are numerically unstable, and they have known inverses. However, it is a mistake 
to compare the known inverse, given in this exercise, to the computed inverse of a 
Hilbert matrix, since the matrix to be inverted must be expressed in rounded numbers 
beforehand; the inverse of an approximate Hilbert matrix will be somewhat different 
from the inverse of an exact one, due to the instability present. Since the elements 
of the inverse are integers, and since the inverse matrix is just as unstable as the 
original, the inverse can be specified exactly, and one could try to invert the inverse. 
The integers that appear in the inverse are, however, quite large.] The solution to this 
problem requires an elementary knowledge of factorials and binomial coefficients, which 
are discussed in Sections 1.2.5 and 1.2.6. 


46. [M30] Let A be an m x n matrix, and let B be an n x m matrix. Given that 
1 < ji, j2,.--, jm <n, let Aj, jo...53, denote the m x m matrix consisting of columns 
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Jiy+++3Jm Of A, and let Bj, 55...;,, denote the m x m matrix consisting of rows j1,...,Jm 
of B. Prove the Binet-Cauchy identity 
det (AB) = 5 det (Aji j2..jm ) det (Bj jo...5m)- 


1<ji<j2 < <jm <N 
(Note the special cases: (i) m = n, (ii) m = 1, (iii) B = AT, (iv) m >n, (v) m = 2.) 
47. [M27] (C. Krattenthaler.) Prove that 
(x+ q2)(x +q) (x+pı)(x +q) («+ pr) 
det 


( ) 
(yt+qa)\(ytas) (y +pı)(y +q) (y+ pı)(y + p2) ) 
(z +q2)\(z +q) (z+pı)(z +q) (z+p)(z ) 


= (x — y)(x — z)(y — z)(pı — q2) (pı — q3)(p2 — q3), 


and generalize this equation to an identity for an n x n determinant in 3n — 2 variables 
U1,-+-; En, P1; ---, Pn—1; Q2, ---, qn. Compare your formula to the result of exercise 38. 


1.2.4. Integer Functions and Elementary Number Theory 


If x is any real number, we write 


|x| = the greatest integer less than or equal to x (the floor of x); 
[x] = the least integer greater than or equal to x (the ceiling of x). 


The notation [|x] was often used before 1970 for one or the other of these 
functions, usually the former; but the notations above, introduced by K. E. 
Iverson in the 1960s, are more useful, because |x| and [x] occur about equally 
often in practice. The function || is sometimes called the entier function, from 
the French word for “integer.” 

The following formulas and examples are easily verified: 


[v2] =1, [v2]=2 [+5 ]=0, [-3]=0, [+5] =—1 (not zcrol); 
fa] = |a| if and only if x is an integer, 


fa] =|a|+1 if and only if x is not an integer; 
|-z|=—-[z]; w-l<|#])<a2< fa] <a+1. 
Exercises at the end of this section list other important formulas involving the 
floor and ceiling operations. 
If x and y are any real numbers, we define the following binary operation: 
xmody=ax-—yl|a/y|, ify #0; xmod0= z. (1) 
From this definition we can see that, when y Æ 0, 
x 


0< | 
y 


z| _ xmody 
y y 


<i. (2) 


Consequently 
a) ify >0, then 0 < z mod y < y; 
b) if y < 0, then 0 > zmody > y; 
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c) the quantity x — (x mod y) is an integral multiple of y. 
We call z mod y the remainder when x is divided by y; similarly, we call |a/y| 
the quotient. 
When z and y are integers, “mod” is therefore a familiar operation: 


5 mod 3 = 2, 18 mod 3 = 0, —2 mod 3 = 1. (3) 


We have x mod y = 0 if and only if x is a multiple of y, that is, if and only if x is 
divisible by y. The notation y\x, read “y divides x,” means that y is a positive 
integer and x mod y = 0. 

The “mod” operation is useful also when x and y take arbitrary real values. 
For example, with trigonometric functions we can write 


tan x = tan (x mod 7). 
The quantity x mod 1 is the fractional part of x; we have, by Eq. (1), 
x = |x] + (x mod 1). (4) 


Writers on number theory often use the abbreviation “mod” in a different 
but closely related sense. We will use the following form to express the number- 
theoretical concept of congruence: The statement 


z= y (modulo z) (5) 


means that x mod z = y mod z; it is the same as saying that x — y is an integral 
multiple of z. Expression (5) is read, “a is congruent to y modulo z.” 

Let’s turn now to the basic elementary properties of congruences that will 
be used in the number-theoretical arguments of this book. All variables in the 
following formulas are assumed to be integers. Two integers x and y are said 
to be relatively prime if they have no common factor, that is, if their greatest 
common divisor is 1; in such a case we write x L y. The concept of relatively 
prime integers is a familiar one, since it is customary to say that a fraction is in 
“lowest terms” when the numerator is relatively prime to the denominator. 


Law A. If a =b and z = y, then a + z = b + y and az = by (modulo m). 


Law B. If ax = by and a = b, and if a L m, then x = y (modulo m). 
Law C. a =b (modulo m) if and only if an = bn (modulo mn), when n Æ 0. 


Law D. If r L s, then a = b (modulo rs) if and only if a = b (modulo r) and 
a = b (modulo s). 


Law A states that we can do addition, subtraction, and multiplication 
modulo m just as we do ordinary addition, subtraction, and multiplication. 
Law B considers the operation of division and shows that, when the divisor 
is relatively prime to the modulus, we can also divide out common factors. Laws 
C and D consider what happens when the modulus is changed. These laws are 
proved in the exercises below. 

The following important theorem is a consequence of Laws A and B. 
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Theorem F (Fermat’s theorem, 1640). If p is a prime number, then a? = a 
(modulo p) for all integers a. 


Proof. If a is a multiple of p, obviously a? = 0 = a (modulo p). So we need 
only consider the case a mod p Æ 0. Since p is a prime number, this means that 
a L p. Consider the numbers 


Omodp, amodp, 2amodp, ..., (p—1)amodp. (6) 


These p numbers are all distinct, for if ax mod p = ay mod p, then by defini- 
tion (5) az = ay (modulo p); hence by Law B, x = y (modulo p). 
Since (6) gives p distinct numbers, all nonnegative and less than p, we see 


that the first number is zero and the rest are the integers 1, 2, ..., p— 1 in some 
order. Therefore by Law A, 

(a)(2a) ... ((p—1)a) =1-2... (p—1) (modulo p). (7) 
Multiplying each side of this congruence by a, we obtain 

a?(1-2...(p—1)) =a(1-2... (p—1)) (modulo p); (8) 
and this proves the theorem, since each of the factors 1, 2, ..., p—1 is relatively 


prime to p and can be canceled by Law B. J 


EXERCISES 
1. [00] What are [1.1], |—1.1], [—1.1], [0.99999], and [lg 35]? 
> 2. [01] What is [|r]]? 
3. [M10] Let n be an integer, and let x be a real number. Prove that 
) læ] <n if and only if z < n; b)n < |z] if and only if n < z; 
c) [a] < nif and only if x < n; d) n < [z] if and only if n < zx; 
) læ] =n if and only if x— 1 <n < x, and if and only ifn <x < n+ 1; 
) [a] =n if and only if x < n < x + 1, and if and only ifn-—l<a<n. 
[These formulas are the most important tools for proving facts about |x| and [z].] 


> 4. [M10] Using the previous exercise, prove that |—«| = — fz]. 


5. [16] Given that x is a positive real number, state a simple formula that expresses 


x rounded to the nearest integer. The desired rounding rule is to produce |x| when 


xzmod1 < z, and to produce [x] when z mod 1 > L. Your answer should be a single 


formula that covers both cases. Discuss the rounding that would be obtained by your 
formula when z is negative. 


> 6. [20] Which of the following equations are true for all positive real numbers x? 
(a) [vlz] | = Lvs]; ©) [VT21] = [væ]; (©) [Vl2]] = [v=]. 

7. [M15] Show that |x| + ly] < |x + y| and that equality holds if and only if 
xmod1+ymod1 < 1. Does a similar formula hold for ceilings? 


8. [00] What are 100 mod 3, 100 mod 7, —100 mod 7, —100 mod 0? 
9. [05] What are 5 mod —3, 18 mod —3, —2 mod —3? 
> 10. [10] What are 1.1 mod 1, 0.11 mod .1, 0.11 mod —.1? 


11. [00] What does “x = y (modulo 0)” mean by our conventions? 
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12. [00] What integers are relatively prime to 1? 


13. [M00] By convention, we say that the greatest common divisor of 0 and n is |n]. 
What integers are relatively prime to 0? 


14. [12] If x mod 3 = 2 and x mod 5 = 3, what is x mod 15? 


15. [10] Prove that z(x mod y) = (zx) mod (zy). [Law C is an immediate consequence 
of this distributive law.] 


16. [M10] Assume that y > 0. Show that if (x — z)/y is an integer and if 0 < z < y, 
then z = «mod y. 


17. [M15] Prove Law A directly from the definition of congruence, and also prove half 
of Law D: If a = b (modulo rs), then a = b (modulo r) and a = b (modulo s). (Here r 
and s are arbitrary integers.) 


18. [M15] Using Law B, prove the other half of Law D: If a = b (modulo r) and a= b 
(modulo s), then a = b (modulo rs), provided that r L s. 


19. [M10] (Law of inverses.) If n L m, there is an integer n’ such that nn’ = 1 
(modulo m). Prove this, using the extension of Euclid’s algorithm (Algorithm 1.2.1E). 


20. [M15] Use the law of inverses and Law A to prove Law B. 


21. [M22] (Fundamental theorem of arithmetic.) Use Law B and exercise 1.2.1—5 to 
prove that every integer n > 1 has a unique representation as a product of primes 
(except for the order of the factors). In other words, show that there is exactly one 
way to write n = pipo...pr, where each p; is prime and pı < po < +- < px. 


22. [M10] Give an example to show that Law B is not always true if a is not relatively 
prime to m. 


23. [M10] Give an example to show that Law D is not always true if r is not relatively 
prime to s. 


24. [M20] To what extent can Laws A, B, C, and D be generalized to apply to 
arbitrary real numbers instead of integers? 

25. [M02] Show that, according to Theorem F, a?~! mod p = [a is not a multiple 
of p], whenever p is a prime number. 


26. [M15] Let p be an odd prime number, let a be any integer, and let b = a?TD/2, 
Show that b mod p is either 0 or 1 or p — 1. [Hint: Consider (b + 1)(b — 1).] 


27. [M15] Given that n is a positive integer, let y(n) be the number of values among 
{0,1,...,n — 1} that are relatively prime to n. Thus (1) = 1, (2) = 1, (3) = 2, 
(4) = 2, etc. Show that y(p) = p — 1 if p is a prime number; and evaluate (p°), 
when e is a positive integer. 


28. [M25] Show that the method used to prove Theorem F can be used to prove the 
following extension, called Euler’s theorem: a?™) =] (modulo m), for any positive 
integer m, when a L m. (In particular, the number n’ in exercise 19 may be taken to 
be n?(™=-1 mod m.) 


29. [M22] A function f(n) of positive integers n is called multiplicative if f(rs) = 
f(r) f(s) whenever r L s. Show that each of the following functions is multiplicative: 
(a) f(n) = n°, where c is any constant; (b) f(n) = [n is not divisible by k? for any 
integer k > 1]; (c) f(n) = c", where k is the number of distinct primes that divide n; 
(d) the product of any two multiplicative functions. 
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30. [M30] Prove that the function y(n) of exercise 27 is multiplicative. Using this 
fact, evaluate (1000000), and give a method for evaluating y(n) in a simple way once 
n has been factored into primes. 

31. [M22] Prove that if f(n) is multiplicative, so is g(n) = Dian f(d). 

32. [M18] Prove the double-summation identity 


> fea] >, > fced), 


an d c\n d\(n/c) 
for any function f(x,y). 


33. [M18] Given that m and n are integers, evaluate (a) | 4(n + m)|+|4(n — m+ 1)|; 
(b) [5(n+m)] + [$(n— m+ 1)]. (The special case m = 0 is worth noting.) 
34. [M21] What conditions on the real number b > 1 are necessary and sufficient to 


guarantee that |log, £] = |log,|2] | for all real x > 1? 


35. [M20] Given that m and n are integers and n > 0, prove that 
L(x + m)/n] = |(lz] +m)/n] 
for all real x. (When m = 0, we have an important special case.) Does an analogous 
result hold for the ceiling function? 
36. [M23] Prove that )77_,|k/2| = [n?/4]; also evaluate X}, [k/2]. 
37. [M30] Let m and n be integers, n > 0. Show that 
= [==] _ (m—-1)(n—-1) d—1 + dlx/d], 


n 
O<k<n 


where d is the greatest common divisor of m and n, and x is any real number. 
38. [M26] (E. Busche, 1909.) Prove that, for all real z and y with y > 0, 


E [e+ Ë] = lev + e+ el -w)]. 


0<k<y 


In particular, when y is a positive integer n, we have the important formula 


|z] 4 [z4 ] fe. 4 [z+ T=] = [ns]. 


n 


39. [HM35] A function f for which f(x) + f(e + 4) +---+ f(x + =) = f(nz), 

whenever n is a positive integer, is called a replicative function. The previous exercise 

establishes the fact that |x] is replicative. Show that the following functions are 

replicative: 

a) f(e)=2—3; 

) f(x) = [zx is an integer]; 

c) f(x) = [x is a positive integer]; 

d) f(x) = [there exists a rational number r and an integer m such that z =rr+m]; 

e) three other functions like the one in (d), with r and/or m restricted to positive 
values; 

f) f(x) = log |2sin 7a|, if the value f(x) = —oo is allowed; 

g) the sum of any two replicative functions; 

h) a constant multiple of a replicative function; 

i) the function g(x) = f(x — |x|), where f(x) is replicative. 


=| 
=| 
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40. [HM46] Study the class of replicative functions; determine all replicative functions 
of a special type. For example, is the function in (a) of exercise 39 the only continuous 
replicative function? It may be interesting to study also the more general class of 
functions for which 


f(z) 4 f(e z) pind f (aṣ Pat) = an f(t) + bn 


Here an and bn are numbers that depend on n but not on x. Derivatives and (if bn = 0) 
integrals of these functions are of the same type. If we require that bn = 0, we have, 
for example, the Bernoulli polynomials, the trigonometric functions cot tx and csc?ra, 
as well as Hurwitz’s generalized zeta function ¢(s,2) = 0,5 1/(k + 2)°* for fixed s. 
With bn 4 0 we have still other well-known functions, such as the psi function. 


Al. [M23] Let a1, a2, a3, ... be the sequence 1, 2, 2, 3, 3, 3, 4, 4, 4, 4, ...; find an 
expression for an in terms of n, using the floor and/or ceiling function. 


42. [M24] (a) Prove that 


n n-1 
5 Qk = Nan — 5 k(ak+ı — ak), if n > 0. 
k=1 k=1 


(b) The preceding formula is useful for evaluating certain sums involving the floor 
function. Prove that, if b is an integer > 2, 


n 


Y Log, k] = (n + 1) [log, n] — (ble +4 — b) /(b— 1). 


k=1 
43. [M23] Evaluate 77_,|Vk J. 
44. [M24] Show that i559 Micjcol(n + jo’) /b***| = n, if b and n are integers, 
n > 0, and b > 2. What is the value of this sum when n < 0? 
45. [M28] The result of exercise 37 is somewhat surprising, since it implies that 
mk+a nk+a 
D=] > |e 


O<k<n O0<k<m 


when m and n are positive integers and x is arbitrary. This “reciprocity relationship” 
is one of many similar formulas (see Section 3.3.3). Show that in general we have 


mj rn 
D (2) E [EF] oe -v-s0y + apem-0 
n m 
n O<r<m 
for any function f and all integers m,n > 0. In particular, prove that 


ee ela ee) 


O0<j<n O<j<m 


[Hint: Consider the change of variable r = |mj/n]. Binomial coefficients (‘7’) are 
discussed in Section 1.2.6.] 


46. [M29] (General reciprocity law.) Extend the formula of exercise 45 to obtain an 
expression for J o<j<an /(Lmj/n]), where a is any positive real number. 
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> 47. [M31] When p is an odd prime number, the Legendre symbol (2) is defined to 
be +1, 0, or —1, depending on whether gD? mod p is 1, 0, or p— 1. (Exercise 26 


proves that these are the only possible values.) 


a) Given that q is not a multiple of p, show that the numbers 
(—1)*4/P|(2kqmodp), O<k<p/2, 


are congruent in some order to the numbers 2, 4, ..., p — 1 (modulo p). Hence 


(2) = (—1)7 where c = Do<k<p/2l2ka/P]- 

b) Use the result of (a) to calculate È). 

c) Given that q is odd, show that ’’o<p<p/2 L2ka/p] = X o<rk<p/2lk4/p] (modulo 2), 
unless q is a multiple of p. [Hint: Consider the quantity |(p — 1 — 2k)q/p].] 

d) Use the general reciprocity formula of exercise 46 to obtain the law of quadratic 


reciprocity, (2) (2) = (-1)®-DU-D/4 given that p and q are distinct odd primes. 


48. [M26] Prove or disprove the following identities, for integers m and n: 


(a) [MARAT] fm), [REAR Ln /P5) | _ | Sard | 


49. [M30] Suppose the integer-valued function f(x) satisfies the two simple laws 
(i) f(a +1) = f(x) +1; (ii) f(x) = f(f(nz)/n) for all positive integers n. Prove 
that either f(x) = |x] for all rational z, or f(x) = [x] for all rational zx. 


1.2.5. Permutations and Factorials 


A permutation of n objects is an arrangement of n distinct objects in a row. 
There are six permutations of three objects {a, b, c}: 


abe, acb, bac, bea, cab, cba. (1) 


The properties of permutations are of great importance in the analysis of 
algorithms, and we will deduce many interesting facts about them later in this 
book.* Our first task is simply to count them: How many permutations of n 
objects are possible? There are n ways to choose the leftmost object, and once 
this choice has been made there are n — 1 ways to select a different object to place 
next to it; this gives us n(n — 1) choices for the first two positions. Similarly, we 
find that there are n — 2 choices for the third object distinct from the first two, 
and a total of n(n — 1)(n — 2) possible ways to choose the first three objects. In 
general, if pnk denotes the number of ways to choose k objects out of n and to 
arrange them in a row, we see that 


Pnk =n(n—1)...(n—k+1). (2) 


The total number of permutations is therefore Pnn = n(n — 1) ... (1). 
The process of constructing all permutations of n objects in an inductive 
manner, assuming that all permutations of n — 1 objects have been constructed, 


* In fact, permutations are so important, Vaughan Pratt has suggested calling them 
“perms.” As soon as Pratt’s convention is established, textbooks of computer science will 
be somewhat shorter (and perhaps less expensive). 
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is very important in our applications. Let us rewrite (1) using the numbers 
{1,2,3} instead of the letters {a,b,c}; the permutations are then 


123, 132, 213, 231, 312, 321. (3) 
Consider how to get from this array to the permutations of {1,2,3,4}. There 
are two principal ways to go from n — 1 objects to n objects. 
Method 1. For each permutation aj a2... an-ı of {1,2,...,n—1}, form n 
others by inserting the number n in all possible places, obtaining 
Ma, a2... An-1; a, nag... An—-1; sey a, ag...NQn-1, a, a2... An—-17N. 


For example, from the permutation 2 3 1 in (3), we get 4231,2431,2341, 
2314. It is clear that all permutations of n objects are obtained in this manner 
and that no permutation is obtained more than once. 


Method 2. For each permutation a1 a2... @n—1 of {1,2,...,n—1}, form n 
others as follows: First construct the array 

a1 AQ... Gn—1 $, araz ... On—1 3, e.g aiaz... Gn—1 (n— 4). 
Then rename the elements of each permutation using the numbers {1,2,..., n}, 


preserving order. For example, from the permutation 2 3 1 in (3) we get 
7 
2314, 2313, 2313, 2315 
and, renaming, we get 
3421, 3412, 2413, 2314. 


Another way to describe this process is to take the permutation aja2...@n—1 
and a number k, 1 < k < n; add one to each a; whose value is > k, thus 
obtaining a permutation b1b2 ...bn—1 of the elements {1,...,k-—1,k+1,...,n}; 
then bıb2...bn—1k is a permutation of {1,...,n}. 

Again it is clear that we obtain each permutation of n elements exactly once 
by this construction. Putting k at the left instead of the right, or putting k in 
any other fixed position, would obviously work just as well. 

If pn is the number of permutations of n objects, both of these methods show 
that pp = Nnpn-1; this offers us two further proofs that pn = n(n — 1) ... (1), as 
we already established in Eq. (2). 

The important quantity p, is called n factorial and it is written 


mal. m= [Tk (4) 
k=1 


Our convention for vacuous products (Section 1.2.3) gives us the value 


Oo! = 1, (5) 


and with this convention the basic identity 
nl=(n—1)!n (6) 


is valid for all positive integers n. 
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Factorials come up sufficiently often in computer work that the reader is 
advised to memorize the values of the first few: 


Ol=1, 1!=1, 2=2, 3!=6, 4!=24, 5!= 120. 


The factorials increase very rapidly; for example, 1000! is an integer with over 
2500 decimal digits. 

It is helpful to keep the value 10! = 3,628,800 in mind; one should remember 
that 10! is about 33 million. In a sense, this number represents an approximate 
dividing line between things that are practical to compute and things that are 
not. If an algorithm requires the testing of more than 10! cases, it may consume 
too much computer time to be practical. On the other hand, if we decide to 
test 10! cases and each case requires, say, one millisecond of computer time, 
then the entire run will take about an hour. These comments are very vague, of 
course, but they can be useful to give an intuitive idea of what is computationally 
feasible. 

It is only natural to wonder what relation n! bears to other quantities in 
mathematics. Is there any way to tell how large 1000! is, without laboriously 
carrying out the multiplications implied in Eq. (4)? The answer was found by 
James Stirling in his famous work Methodus Differentialis (1730), page 137; we 
have 


n! = V2rn Ey. (7) 


The “~” sign that appears here denotes “approximately equal,” and “e” is the 
base of natural logarithms introduced in Section 1.2.2. We will prove Stirling’s 
approximation (7) in Section 1.2.11.2. Exercise 24 gives a simple proof of a less 
precise result. 

As an example of the use of this formula, we may compute 


8 
40320 = 8! ~ 4T (=) = 226/7 e78 ~ 67108864 - 1.77245 -0.00033546 ~ 39902. 
E 


In this case the error is about 1%; we will see later that the relative error is 
approximately 1/(12n). 

In addition to the approximate value given by Eq. (7), we can also rather 
easily obtain the exact value of n! factored into primes. In fact, the prime p is a 
divisor of n! with the multiplicity 


eege ge 8 


For example, if n = 1000 and p = 3, we have 
T Ey | po . Ea | = | H = 
H= |z | Lg 27 | L81 | L243) L729 


= 333 + 111 + 37 + 12 + 4 + 1 = 498, 


so 1000! is divisible by 34°° but not by 349°. Although formula (8) is written as an 
infinite sum, it is really finite for any particular values of n and p, because all of 
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the terms are eventually zero. It follows from exercise 1.2.4-35 that |n/p*t!| = 
| [n/p*|/p|; this fact facilitates the calculation in Eq. (8), since we can just 
divide the value of the previous term by p and discard the remainder. 

Equation (8) follows from the fact that |n/p*| is the number of integers 
among {1,2,...,n} that are multiples of p*. If we study the integers in the 
product (4), any integer that is divisible by pî but not by p+! is counted exactly 
j times: once in |n/p|, once in |n/p?|, ..., once in |n/p?|. This accounts for 
all occurrences of p as a factor of n!. [See A. M. Legendre, Essai sur la Théorie 
des Nombres, second edition (Paris: 1808), page 8.] 

Another natural question arises: Now that we have defined n! for non- 
negative integers n, perhaps the factorial function is meaningful also for rational 
values of n, and even for real values. What is (4) l, for example? Let us illustrate 
this point by introducing the “termial” function 


n?=1+2+ = +n= Xk, (9) 
k=1 


which is analogous to the factorial function except that we are adding instead 
of multiplying. We already know the sum of this arithmetic progression from 
Eq. 1.2.3-(15): 
n? = in(n +1). (10) 

This suggests a good way to generalize the “termial” function to arbitrary n, 
by using (10) instead of (9). We have (4)? = 2. 

Stirling himself made several attempts to generalize n! to noninteger n. He 
extended the approximation (7) into an infinite sum, but unfortunately the sum 
did not converge for any value of n; his method gave extremely good approxi- 
mations, but it couldn’t be extended to give an eract value. [For a discussion 
of this somewhat unusual situation, see K. Knopp, Theory and Application of 
Infinite Series, 2nd ed. (Glasgow: Blackie, 1951), 518-520, 527, 534.] 

Stirling tried again, by noticing that 


n=1+(1 i)" (: i yn 4 
(1 L | l a)nn ee ai 


(We will prove this formula in the next section.) The apparently infinite sum 
in Eq. (11) is in reality finite for any nonnegative integer n; however, it does 
not provide the desired generalization of n!, since the infinite sum does not exist 
except when n is a nonnegative integer. (See exercise 16.) 


Still undaunted, Stirling found a sequence aj, a2,... such that 
Inn! = ayn + agn(n—1) +--+ = Soars Il (n— j). (12) 
k>0 O<j<k 


He was unable to prove that this sum defined n! for all fractional values of n, 
although he was able to deduce the value of (4)! = /7/2. 
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At about the same time, Leonhard Euler considered the same problem, and 
he was the first to find the appropriate generalization: 
mm! 


me bs FIF) (am) 5 


Euler communicated this idea in a letter to Christian Goldbach on October 13, 
1729. His formula defines n! for any value of n except negative integers (when 
the denominator becomes zero); in such cases n! is taken to be infinite. Exercises 
8 and 22 explain why Eq. (13) is a reasonable definition. 

Nearly two centuries later, in 1900, C. Hermite proved that Stirling’s idea 
(12) actually does define n! successfully for nonintegers n, and that in fact Euler’s 
and Stirling’s generalizations are identical. 

Many notations were used for factorials in the early days. Euler actually 
wrote [n], Gauss wrote JI n, and the symbols |n and n| were popular in England 
and Italy. The notation n!, which is universally used today when n is an inte- 
ger, was introduced by a comparatively little known mathematician, Christian 
Kramp, in an algebra text [Elémens d’Arithmétique Universelle (Cologne: 1808), 
page 219]. 

When n is not an integer, however, the notation n! is less common; instead 
we customarily employ a notation due to A. M. Legendre: 


n!=I(n+1)=nI (n). (14) 


This function I(x) is called the gamma function, and by Eq. (13) we have the 
definition 


nes = a H1) (+2)... (+m) (15) 
A graph of I(x) is shown in Fig. 7. 

Equations (13) and (15) define factorials and the gamma function for com- 
plex values as well as real values; but we generally use the letter z, instead of n 
or x, when thinking of a variable that has both real and imaginary parts. The 
factorial and gamma functions are related not only by the rule z! = P'(z+1) but 
also by 


T 


(—z)!P(z) = - 3 (16) 
sin Tz 

which holds whenever z is not an integer. (See exercise 23.) 

Although T(z) is infinite when z is zero or a negative integer, the function 

1/I (z) is well defined for all complex z. (See exercise 1.2.7-24.) Advanced 

applications of the gamma function often make use of an important contour 


integral formula due to Hermann Hankel: 


1 1 i T 
T(z) 2ri p?’ a 


the path of complex integration starts at —oo, then circles the origin in a 
counterclockwise direction and returns to —oo. [Zeitschrift fiir Math. und Physik 
9 (1864), 1-21.] 
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5 


6 
—4 -3 -2 -1 0 1 2 3 4 


Fig. 7. The function I(x) = (a —1)!. The local minimum at X has the coordinates 
(1.46163 21449 68362 34126 26595, 0.88560 31944 10888 70027 88159). 


Many formulas of discrete mathematics involve factorial-like products known 
as factorial powers. The quantities zë and x* (read, “x to the k falling” and “x 
to the k rising”) are defined as follows, when k is a positive integer: 


k-1 
ak = g(x- 1)... (x-k +1) = |[(@-5); (18) 
j=0 
i k—1 
zt = 2(x+1)...(e+k-1) = |] (e+). (19) 
j=0 
Thus, for example, the number ppg of (2) is just në. Notice that we have 
zë = (x+k—1)8 = (—-1)*(-a)é. (20) 
The general formulas 
zë = l gk = P(x +k) (21) 
(x-k)? T(x) 


can be used to define factorial powers for other values of k. [The notations z” and 
xË are due respectively to A. Capelli, Giornale di Mat. di Battaglini 31 (1893), 
291-313, and L. Toscano, Comment. Accademia della Scienze 3 (1939), 721—757.] 

The interesting history of factorials from the time of Stirling to the present 
day is traced in an article by P. J. Davis, “Leonhard Euler’s integral: A historical 
profile of the gamma function,” AMM 66 (1959), 849-869. See also J. Dutka, 
Archive for History of Exact Sciences 31 (1984), 15-34. 
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EXERCISES 

1. [00] How many ways are there to shuffle a 52-card deck? 

2. [10] In the notation of Eq. (2), show that ppin—1) = Pnn, and explain why this 
happens. 


3. [10] What permutations of {1,2,3,4,5} would be constructed from the permuta- 
tion 3 1 2 4 using Methods 1 and 2, respectively? 


4. [13] Given the fact that log,) 1000! = 2567.60464..., determine exactly how 
many decimal digits are present in the number 1000!. What is the most significant 
digit? What is the least significant digit? 


5. [15] Estimate 8! using the following more exact version of Stirling’s approxi- 


mation: 1 
n n 
ly V2 — 1 z) 
n gi ( r) ( 7 12n 


6. [17] Using Eq. (8), write 20! as a product of prime factors. 


7. [M10] Show that the “generalized termial” function in Eq. (10) satisfies the 
identity x? = x + (x — 1)? for all real numbers zx. 


8. [HM15] Show that the limit in Eq. (13) does equal n! when n is a nonnegative 
integer. 

9. [M10] Determine the values of (3) and I'(—3), given that (4)! = /7/2. 
10. [HM20] Does the identity T(x +1) = xI(x) hold for all real numbers x? (See 
exercise 7.) 


11. [M15] Let the representation of n in the binary system be n = 2°1+2°2+.--4+2°7, 
where e1 > e2 >--: > er > 0. Show that n! is divisible by 2"~" but not by grarrk, 


12. [M22] (A. Legendre, 1808.) Generalizing the result of the previous exercise, let 
p be a prime number, and let the representation of n in the p-ary number system be 
n = akp? + ag_ip* +--+ aip + ao. Express the number pz of Eq. (8) in a simple 
formula involving n, p, and a’s. 

13. [M23] (Wilson’s theorem, actually due to Leibniz, 1682.) If p is prime, then 
(p — 1)!mod p = p—1. Prove this, by pairing off numbers among {1,2,..., p — 1} 
whose product modulo p is 1. 

14. [M28] (L. Stickelberger, 1890.) In the notation of exercise 12, we can determine 
n! mod p in terms of the p-ary representation, for any positive integer n, thus general- 
izing Wilson’s theorem. In fact, prove that n!/p“ = (—1)"ao! ai!...ax! (modulo p). 


15. [HM15] The permanent of a square matrix is defined by the same expansion as 
the determinant except that each term of the permanent is given a plus sign while the 
determinant alternates between plus and minus. Thus the permanent of 


a b c 
d e f 
g h i 
is aei + bfg + cdh + gec + hfa + idb. What is the permanent of 
1x1 1x2 ... 1xn 


2x1 2x2 ane 2X N 


nmxlnx2 ... nxn 
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16. [HM15] Show that the infinite sum in Eq. (11) does not converge unless n is a 
nonnegative integer. 


17. [HM20] Prove that the infinite product 


(n+ a1)... (n+ ax) 
II (n + fi)... (n + Be) 


n>1 
equals (1+ 61)... 114+ Be)/PA+a1)... T(1+ ar), if a1 +- +ak = b1 ++ Br 
and if none of the ’s is a negative integer. 
18. [M20] Assume that 7/2 = 2.2.4.4.6.8..... (This is “Wallis’s product,” 


obtained by J. Wallis in 1655, and we will prove it in exercise 1.2.6—43.) Using the 
previous exercise, prove that (4)! = yT/2. 
19. [HM22] Denote the quantity appearing after “limm—.” in Eq. (15) by Im(2). 
Show that 
a t\™ x—1 x 2 mxr—l1 : 
fwe (1- ~) odt=m? | 0-9 dt, ife>0. 
0 m 0 
20. [HM21] Using the fact that 0 < e™ — (1—t/m)™ < t?e~*/m, if 0 < t < m, and 
the previous exercise, show that I(x) = ie e't®—1 dt, if a > 0. 
21. [HM25] (L. F. A. Arbogast, 1800.) Let D%u represent the kth derivative of a 
function u with respect to x. The chain rule states that Diw = Diw Dzu. If we apply 
this to second derivatives, we find D?w = D2w(Diu)? + Diw D2u. Show that the 
general formula is 
ik . ! 
n — j n. 1 kı n kn 
Dzw=)_ 5 Diwyg bal yee Pe (Dru. 


I=0 ki+kə++kn=j 
kyi +2koq+--+nkyn=n 
ky ,k2,...,kn >0 


22. [HM20] Try to put yourself in Euler’s place, looking for a way to generalize n! 
to noninteger values of n. Since (n + 4)!/n! times ((n + $) + 4)!/(n + 4)! equals 
(n + 1)!/n! = n +1, it seems natural that (n + 4)!/n! should be approximately yn. 
Similarly, (n+ 4)!/n! should be ~ Y/n. Invent a hypothesis about the ratio (n + x)!/n! 
as n approaches infinity. Is your hypothesis correct when x is an integer? Does it tell 
anything about the appropriate value of x! when x is not an integer? 

23. [HM20] Prove (16), given that mz [|X (1 — 2?/n”) = sin 7z. 

24. [HM21]| Prove the handy inequalities 


n” nt 


ace ! es i 
aa <n! < Se integer n > 1. 


[Hint: 1+ x < e” for all real x; hence (k + 1)/k < e"! < k/(k—1),] 


25. [M20] Do factorial powers satisfy a law analogous to the ordinary law of expo- 
nents, r+” = 4a"? 


1.2.6. Binomial Coefficients 


The combinations of n objects taken k at a time are the possible choices of 
k different elements from a collection of n objects, disregarding order. The 
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combinations of the five objects {a, b,c, d,e} taken three at a time are 
abc, abd, abe, acd, ace, ade, bcd, bce, bde, cde. (1) 


It is a simple matter to count the total number of k-combinations of n objects: 
Equation (2) of the previous section told us that there are n(n—1)...(n—k+1) 
ways to choose the first k objects for a permutation; and every k-combination 
appears exactly k! times in these arrangements, since each combination appears 
in all its permutations. Therefore the number of combinations, which we denote 


by ‘ea? is 


n\  n(n-—1)...(n-k+1) 
(x)= k(k—-l...d) ` (2) 


G)=s55 = 10 
3 3-2.1 


which is the number of combinations we found in (1). 

The quantity (g), read “n choose k,” is called a binomial coefficient; these 
numbers have an extraordinary number of applications. They are probably the 
most important quantities entering into the analysis of algorithms, so the reader 
is urged to become familiar with them. 

Equation (2) may be used to define (7) even when n is not an integer. To 
be precise, we define the symbol (7) for all real numbers r and all integers k as 
follows: 

A r(r—1)...(r—k+1)_ rE 


: r+1l— j 
= = = - , integer k > 0; 
k(k—1)...(1) k! lI j 5 


For example, 


P (3) 
( ) =0, integer k < 0. 


In particular cases we have 


Q- Qe @-2 v 


Table 1 gives values of the binomial coefficients for small integer values of r 
and k; the values for 0 < r < 4 should be memorized. 

Binomial coefficients have a long and interesting history. Table 1 is called 
“Pascal’s triangle” because it appeared in Blaise Pascal’s Traité du Triangle 
Arithmétique in 1653. This treatise was significant because it was one of the 
first works on probability theory, but Pascal did not invent the binomial co- 
efficients (which were well-known in Europe at that time). Table 1 also ap- 
peared in the treatise Szu-yüan Yü-chien (“The Precious Mirror of the Four 
Elements”) by the Chinese mathematician Chu Shih-Chieh in 1303, where they 
were said to be an old invention. Yang Hui, in 1261, credited them to Chia 
Hsien (c. 1100), whose work is now lost. The earliest known detailed discussion 
of binomial coefficients is in a tenth-century commentary, due to Halayudha, on 
an ancient Hindu classic, Pingala’s Chandahsastra. [See G. Chakravarti, Bull. 
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Table 1 

TABLE OF BINOMIAL COEFFICIENTS (PASCAL’S TRIANGLE) 

Tr r r Tr T r y r r r 
á (5) ee io) (3) a (5) (5) (a) ” (3) 
of 1 o o o o 0 0 0 0 o 
ili 1 0 0 0 0 0 0 0 0 
21 2 1 0 0 0 0 0 0 0 
31 3 3 1 0 0 0 0 0 0 
41 4 6 4 1 0 0 0 0 0 
5) 1 5 10 10 5 1 0 0 0 0 
61 6 135 20 mb 6 1 0 0 0 
71 7 A 3 3 2 7 1 0 0 
8| 1 8 28 56 70 56 2 8 1ı 0 
9| 1 9 36 84 12 16 84 36 9 1 


Calcutta Math. Soc. 24 (1932), 79-88.) Another 
Indian mathematician, Mahavira, had previously 
explained rule (3) for computing (7) in Chapter 6 
of his Ganita Sara Sangraha, written about 850; 
and in 1150 Bhaskara repeated Mahavira’s rule 
near the end of his famous book Lilavati. For small 
values of k, binomial coefficients were known much 
earlier; they appeared in Greek and Roman writ- 
ings with a geometric interpretation (see Fig. 8). 
The notation (o) was introduced by Andreas von 
Ettingshausen in §31 of his book Die combina- 
torische Analysis (Vienna: 1826). 

The reader has probably noticed several in- 
teresting patterns in Table 1. Binomial coeffi- 
cients satisfy literally thousands of identities, and 
for centuries their amazing properties have been 
continually explored. In fact, there are so many 
relations present that when someone finds a new Fig. 8. Geometric interpre- 
identity, not many people get excited about it any tation of (2), n=4. 
more, except the discoverer. In order to manipu- 
late the formulas that arise in the analysis of algorithms, a facility for handling 
binomial coefficients is a must, and so an attempt has been made in this section 
to explain in a simple way how to maneuver with these numbers. Mark Twain 
once tried to reduce all jokes to a dozen or so primitive kinds (farmer’s daughter, 
mother-in-law, etc.); we will try to condense the thousands of identities into a 
small set of basic operations with which we can solve nearly every problem 
involving binomial coefficients that we will meet. 

In most applications, both of the numbers r and k that appear in (o) will 
be integers, and some of the techniques we will describe are applicable only in 
such cases. Therefore we will be careful to list, at the right of each numbered 
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equation, any restrictions on the variables that appear. For example, Eq. (3) 
mentions the requirement that k is an integer; there is no restriction on r. The 
identities with fewest restrictions are the most useful. 

Now let us study the basic techniques for operating on binomial coefficients: 


A. Representation by factorials. From Eq. (3) we have immediately 
l 
(J = Haar integer n > integer k > 0. (5) 


This allows combinations of factorials to be represented as binomial coefficients 
and conversely. 


B. Symmetry condition. From Eqs. (3) and (5), we have 


a = ( X |: integer n > 0, integer k. (6) 
k n—k 


This formula holds for all integers k. When k is negative or greater than n, the 
binomial coefficient is zero (provided that n is a nonnegative integer). 


C. Moving in and out of parentheses. From the definition (3), we have 


6 ae integer k Æ 0. (7) 


This formula is very useful for combining a binomial coefficient with other parts 
of an expression. By elementary transformation we have the rules 


r r—1 l/r 1l/r-1 
i (;.) ee. i) ae), = ie i): 
the first of which is valid for all integers k, and the second when no division by 
zero has been performed. We also have a similar relation: 


OERS ipa integer k A r. (8) 


Let us illustrate these transformations, by proving Eq. (8) using Eqs. (6) 
and (7) alternately: 


(;.) ~ er =h, ILa) n ): 


[Note: This derivation is valid only when r is a positive integer 4 k, because of 
the constraints involved in Eqs. (6) and (7); yet Eq. (8) claims to be valid for 
arbitrary r # k. This can be proved in a simple and important manner: We 


hav ev erified that 
k k 


for infinitely many values of r. Both sides of this equation are polynomials in r. 
A nonzero polynomial of degree n can have at most n distinct zeros; so (by 
subtraction) if two polynomials of degree < n agree at n +1 or more different 
points, the polynomials are identically equal. This principle may be used to 
extend the validity of many identities from integers to all real numbers.] 
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D. Addition formula. The basic relation 


Lee JP) integer k, (9) 


is clearly valid in Table 1 (every value is the sum of the two values above and 
to the left) and we may easily verify it in general from Eq. (3). Alternatively, 
Eqs. (7) and (8) tell us that 


PO ea) PNG) PEG) eG), 


Equation (9) is often useful in obtaining proofs by induction on r, when r is an 
integer. 


E. Summation formulas. Repeated application of (9) gives 
OG Ge Ga) 
ki \k k-1/ \k k-1 k-2) 0’ 
Gea er) e 
k/ \k-1 k / \k-1 k-1 kj o 


Thus we are led to two important summation formulas that can be expressed as 


follows: 
RR ee eC 


integer n >0. (10) 


Se) ee 


integer m > 0, integer n> 0. (11) 


or 


Equation (11) can easily be proved by induction on n, but it is interesting 
to see how it can also be derived from Eq. (10) with two applications of Eq. (6): 
k m+k mt+k m+k 
ea a a ER, 
O<k<n O0<m+k<n —m<k<0 0<k<n-—m 
— 1 1 
ae ae m) + ) _ > | 
n-m m+1 
assuming that n > m. If n < m, Eq. (11) is obvious. 
Equation (11) occurs very frequently in applications; in fact, we have already 
derived special cases of it in previous sections. For example, when m = 1, we 
have our old friend, the sum of an arithmetic progression: 


() (i) er () emetaenann (Mt) = eae 
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Suppose that we want a simple formula for the sum 1? + 2?+---+n?. This 
can be obtained by observing that k? = 2(5) + (£); hence 


d= (2) +(2)) =20°S) + (72) 


And this answer, obtained in terms of binomial coefficients, can be put back into 
polynomial notation if desired: 


1 —1 1 
(n+1)n(n ) (n+ LEM | 1) (ny ey: is) 
6 2 
The sum 13 +2 +- -+n can be obtained in a similar way; any polynomial 
ao + aik + agk? +--+ +amk™ can be expressed as bo (6) + bi (F) +--+ + bm (E) 
for suitably chosen coefficients bo, ...,bm. We will return to this subject later. 


12742? 4+.-.4n?=2 


F. The binomial theorem. Of course, the binomial theorem is one of our 
principal tools: 


(z +y) = 2A integer r > 0. (13) 
k 


For example, (x + y)* = xf + 4r°y + 627? + 4ry? + y*. (At last we are able to 
justify the name “binomial coefficient” for the numbers (;).) 

It is important to notice that we have written }°, in Eq. (13), rather than 
Py as might have been expected. If no restriction is placed on k, we are 
summing over all integers, —oo < k < +00; but the two notations are exactly 
equivalent in this case, since the terms in Eq. (13) are zero when k < 0 or 
k >r. The simpler form 5°, is to be preferred, since all manipulations with 
sums are simpler when the conditions of summation are simpler. We save a 
good deal of tedious effort if we do not need to keep track of the lower and/or 
upper limits of summation, so the limits should be left unspecified whenever 
possible. Our notation has another advantage also: If r is not a nonnegative 
integer, Eq. (13) becomes an infinite sum, and the binomial theorem of calculus 
states that Eq. (13) is valid for all r, if |x/y| < 1. 

It should be noted that formula (13) gives 


0° = 1. (14) 


We will use this convention consistently. 
The special case y = 1 in Eq. (13) is so important, we state it specially: 


yi? =(1+=2)’, integer r > 0 or |z| < 1. (15) 

k 
The discovery of the binomial theorem was announced by Isaac Newton in 
letters to Oldenburg on June 13, 1676 and October 24, 1676. [See D. Struik, 
Source Book in Mathematics (Harvard Univ. Press, 1969), 284-291.] But he 
apparently had no real proof of the formula; at that time the necessity for rigorous 
proof was not fully realized. The first attempted proof was given by L. Euler 
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in 1774, although his effort was incomplete. Finally, C. F. Gauss gave the first 
actual proof in 1812. In fact, Gauss’s work represented the first time anything 
about infinite sums was proved satisfactorily. 

Early in the nineteenth century, N. H. Abel found a surprising generalization 
of the binomial formula (13): 


(z +y)” = DAE —kz)*(y+kz)"-*, integer n> 0, x#0. (16) 

k 
This is an identity in three variables, x, y, and z (see exercises 50 through 52). 
Abel published and proved this formula in Volume 1 of A. L. Crelle’s soon- 
to-be-famous Journal fiir die reine und angewandte Mathematik (1826), pages 
159-160. It is interesting to note that Abel contributed many other papers to the 
same Volume 1, including his famous memoirs on the unsolvability of algebraic 


equations of degree 5 or more by radicals, and on the binomial theorem. See 
H. W. Gould, AMM 69 (1962), 572, for a number of references to Eq. (16). 


G. Negating the upper index. The basic identity 


(o) = (1 (* cs D), integer k, (17) 


follows immediately from the definition (3) when each term of the numerator is 
negated. This is often a useful transformation on the upper index. 
One easy consequence of Eq. (17) is the summation formula 


Jer ~ o 7 a A (-1)"(") = =r a integer n. 
e (18) 


This identity could be proved by induction using Eq. (9), but we can use Eqs. (17) 
and (10) directly: 


EMEC =P) e903) 


Another important application of Eq. (17) can be made when r is an integer: 


@ =(-yen ee ”) , integer n > 0, integer m. (19) 


m n= m 


(Set r = n and k = n — m in Eq. (17) and use (6).) We have moved n from the 
upper position to the lower. 


H. Simplifying products. When products of binomial coefficients appear, they 
can usually be reexpressed in several different ways by expanding into factorials 
and out again using Eq. (5). For example, 


CCE) = (i) Cmi). ieem, integere. (ao) 
m) \ k) =k) (mg) imbeser mm, integer k. 20 


1.2.6 BINOMIAL COEFFICIENTS 59 


It suffices to prove Eq. (20) when r is an integer > m (see the remarks after 
Eq. (8)), and when 0 < k < m. Then 


(n) (a) ml! at (m—k)!  k!(r Sita (r—m)! (;,) ae: 


Equation (20) is very useful when an index (namely m) appears in both the 
upper and the lower position, and we wish to have it appear in one place rather 
than two. Notice that Eq. (7) is the special case of Eq. (20) when k = 1. 


I. Sums of products. To complete our set of binomial-coefficient manipu- 
lations, we present the following very general identities, which are proved in 
the exercises at the end of this section. These formulas show how to sum over a 
product of two binomial coefficients, considering various places where the running 
variable k might appear: 


H(i) (a2) = CS); integer n. (21) 
lea) ow) ~ Cae, 


integer m, integer n, integer r > 0. (22) 


k 
a) ( i jop = ( i ). integer n, integer r > 0. (23) 
- k n n—r 


integer t > 0, integer r > 0, integer m > 0. (24) 


ae = eee 
m n ~\m+tn+ti1/’ 


integer n > integer s > 0, integer m > 0, integer r > 0. (25) 


Den a a aere 160 


k>0 


k=0 


Of these identities, Eq. (21) is by far the most important, and it should be 
memorized. One way to remember it is to interpret the right-hand side as the 
number of ways to select n people from among r men and s women; each 
term on the left is the number of ways to choose k of the men and n — k 
of the women. Equation (21) is commonly called Vandermonde’s convolution, 
since A. Vandermonde published it in Mém. Acad. Roy. Sciences (Paris, 1772), 
part 1, 489-498. However, it had appeared already in Chu Shih-Chieh’s 1303 
treatise mentioned earlier [see J. Needham, Science and Civilisation in China 3 
(Cambridge University Press, 1959), 138-139]. 
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If r = tk in Eq. (26), we avoid the zero denominator by canceling with 
a factor in the numerator; therefore Eq. (26) is a polynomial identity in the 
variables r, s, t. Obviously Eq. (21) is a special case of Eq. (26) with t = 0. 

We should point out a nonobvious use of Eqs. (23) and (25): It is often 
helpful to replace the simple binomial coefficient on the right-hand side by the 
more complicated expression on the left, interchange the order of summation, 
and simplify. We may regard the left-hand sides as expansions of 


S : s+k 
( ) in terms of ( ). 
n+a n 


Formula (23) is used for negative a, formula (25) for positive a. 

This completes our study of binomial-coefficientology. The reader is advised 
to learn especially Eqs. (5), (6), (7), (9), (13), (47), (20), and (21) —frame them 
with your favorite highlighter pen! 

With all these methods at our disposal, we should be able to solve almost 
any problem that comes along, in at least three different ways. The following 
examples illustrate the techniques. 

Example 1. When r is a positive integer, what is the value of Du ) (he 
Solution. Formula (7) is useful for disposing of the outside k: b 


DAO e aa 


Now formula (22) applies, with m = 0 and n = —1. The answer is therefore 
—1 
SOG A e 


k\ (2k\ (-1)* 

Example 2. What is the value of > y z ) ( ) Cur if n is a nonnegative 
2 k/k+1 

integer’ 


Solution. This problem is tougher; the summation index k appears in six places! 
First we apply Eq. (20), and we obtain 


z k k/k+1 
We can now breathe more easily, since several of the menacing characteristics 


of the original formula have disappeared. The next step should be obvious; we 
apply Eq. (7) in a manner similar to the technique used in Example 1: 


De Goer er) 


Good, another k has vanished. At this point there are two equally promising 
lines of attack. We can replace the (T by 2"), assuming that k > 0, and 
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evaluate the sum with Eq. (23): 


S(T HS 


k>0 


jæi w\ 


II 
| 
z 
H| = 
= 
M 
ATR 
z 
3 = 
> 
NY 
7 ™~ 
3 
> 
KH 
NY 
Fait 
pi 
S 
> 


oe ge ee 


Eo 1 c(t) + 1 ~ 1 On), 
n+1 —1 n+1 n n+1 n 
The binomial coefficient (75) equals zero except when n = 0, in which case it 


equals one. So we can conveniently state the answer to our problem as [n =0], 
using Iverson’s convention (Eq. 1,2.3-(16)), or as Ono, using the Kronecker delta 


(Eq. 1.2.3-(19)). 
Another way to proceed from Eq. (27) is to use Eq. (17), obtaining 


er) aa 


k 


We can now apply Eq. (22), which yields the sum 
Cc. (n+ a 1 a 1 
n+1—1+0 /n+1 \n/n+1 
Once again we have derived the answer: 


n+k\ (2k\ (—1)* 
= rn>o0. 
> ( ok )( ie ) ai On0s integer n > 0 (28) 


k 2k\ (—1)* 

Example 3. What is the value of > ( a ) ( ) Cur for positive inte- 
> m+2k k/k+1 

gers m and n? 


Solution. If m were zero, we would have the same formula to work with that 
we had in Example 2. However, the presence of m means that we cannot even 
begin to use the method of the previous solution, since the first step there was 
to use Eq. (20)— which no longer applies. In this situation it pays to complicate 
things even more by replacing the unwanted ee Toe) by a sum of terms of the 
form ig vg since our problem will then become a sum of problems that we know 


2k 
how to solve. Accordingly, we use Eq. (25) with 


r=n+k—-1, m=2k, s=0, n=m-1, 


and we have 


ap a 109s = 


k O<j<n+k-1 
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We wish to perform the summation on k first; but interchanging the order of 
summation demands that we sum on the values of k that are > 0 and > 7—n+1. 
Unfortunately, the latter condition raises problems, because we do not know the 
desired sum if j > n. Let us save the situation, however, by observing the terms 
of (29) are zero when n < j < n+k-—1. This condition implies that k > 1; thus 
0<n+k-1-—j<k-1 < 2k, and the first binomial coefficient in (29) will 
vanish. We may therefore replace the condition on the second sum by 0 < j < n, 
and the interchange of summation is routine. Summing on k by Eq. (28) now 


gives 
j 
3 a = i) Aep: 


O0<j<n 


and all terms are zero except when j = n — 1. Hence our final answer is 


n—-1 
w -1 ) l 
The solution to this problem was fairly complicated, but not really myste- 
rious; there was a good reason for each step. The derivation should be studied 
closely because it illustrates some delicate maneuvering with the conditions in 
our equations. There is actually a better way to attack this problem, however; 


it is left to the reader to figure out a way to transform the given sum so that 
Eq. (26) applies (see exercise 30). 


Example 4. Prove that 


5 Aplr, t)An-k(8s,t) = An(r +s,t), integer n > 0, (30) 
k 


where An (x,t) is the nth degree polynomial in x that satisfies 


re for x £ nt. 


n 


oe) x 
r-n 


Alane ( 


Solution. We may assume that r 4 kt 4 s for 0 < k < n, since both sides of 
(30) are polynomials in r, s, t. Our problem is to evaluate 


CC a”) aa 


k 


which, if anything, looks much worse than our previous horrible problems! Notice 
the strong similarity to Eq. (26), however, and also note the case t = 0. 
We are tempted to change 


on r n a 
O pa 
k r— kt k-1 k’ 
except that the latter tends to lose the analogy with Eq. (26) and it fails when 
k = 0. A better way to proceed is to use the technique of partial fractions, 
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whereby a fraction with a complicated denominator can often be replaced by a 
sum of fractions with simpler denominators. Indeed, we have 


1 1 i 1 
r—kt s—(n—k)t r+s—nt \r-kt s—(n-k)t)` 


Putting this into our sum we get 
s D ars r 
r+s—nt k n—k r — kt 


2 
+ at a) Ca 5 


k 


and Eq. (26) evaluates both of these formulas if we change k to n — k in the 
second; the desired result follows immediately. Identities (26) and (30) are due 
to H. A. Rothe, Formule de Serierum Reversione (Leipzig: 1793); special cases of 
these formulas are still being “discovered” frequently. For the interesting history 
of these identities and some generalizations, see H. W. Gould and J. Kaucky, 
Journal of Combinatorial Theory 1 (1966), 233-247. 
Example 5. Determine the values of ao, a1, a2, ... such that 

n! = ao + ain + agn(n — 1) + agn(n — 1)(n— 2) 4+--- (31) 
for all nonnegative integers n. 
Solution. Equation 1.2.5-(11), which was presented without proof in the previous 
section, gives the answer. Let us pretend that we don’t know it yet. It is clear 
that the problem does have a solution, since we can set n = 0 and determine apo, 
then set n = 1 and determine ay, etc. 

First we would like to write Eq. (31) in terms of binomial coefficients: 


n=O (elar (32) 


k 


The problem of solving implicit equations like this for ap is called the inversion 
problem, and the technique we shall use applies to similar problems as well. 
The idea is based on the special case s = 0 of Eq. (23): 


34 @ (-1)" * = ( . ) = nr, integer n, integer r > 0. (33) 


n n—-T 
k 


The importance of this formula is that when n Æ r, the sum is zero; this enables 
us to solve our problem since a lot of terms cancel out as they did in Example 3: 


Em (Mra Dira) 
“Sea TG) ar 


= k!akôkm = M! am. 
k 
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Notice how we were able to get an equation in which only one value am appears, 


by adding together suitable multiples of Eq. (32) for n = 0, 1,.... We have now 
m—n n! (m — (7r — (—1)” 
am = 3 (1) “(a= » (m-n)! 3 n! 
n>0 O0<n<m O0<n<m 


This completes the solution to Example 5. Let us now take a closer look at 
the implications of Eq. (33): When r and m are nonnegative integers we have 


woo" (@(5) +a(7) tetona) =A 


since the other terms vanish after summation. By properly choosing the coeffi- 
cients c;, we can represent any polynomial in k as a sum of binomial coefficients 
with upper index k. We find therefore that 
DC) (—1)"-¥ (bo + bikt tbk) =rlbp, integrr>0, (34) 
k 

where bo + --- + brk” represents any polynomial whatever of degree r or less. 
(This formula will be of no great surprise to students of numerical analysis, 
since $} (;)(—1)"-* f(x + k) is the “rth difference” of the function f().) 

Using Eq. (34), we can immediately obtain many other relations that appear 
complicated at first and that are often given very lengthy proofs, such as 


Xi) Oo) =t", integer r>0. (35) 


It is customary in textbooks such as this to give a lot of impressive examples 
of neat tricks, etc., but never to mention simple-looking problems where the 
techniques fail. The examples above may have given the impression that all 
things are possible with binomial coefficients; it should be mentioned, however, 
that in spite of Eqs. (10), (11), and (18), there seems to be no simple formula 
for the analogous sum 


Been ee cacy 0 


when n < m. (For n = m the answer is simple; what is it? See exercise 36.) 
On the other hand this sum does have a closed form as a function of n when 
m is an explicit negative integer; for example, 


Ser] or 


k=0 


There is also a simple formula 


DoE e O 8) 


k=0 
for a sum that looks as though it should be harder, not easier. 
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How can we decide when to stop working on a sum that resists simplification? 
Fortunately, there is now a good way to answer that question in many important 
cases: An algorithm due to R. W. Gosper and D. Zeilberger will discover closed 
forms in binomial coefficients when they exist, and will prove the impossibility 
when they do not exist. The Gosper—Zeilberger algorithm is beyond the scope 
of this book, but it is explained in CMath §5.8. See also the book A=B by 
Petkovšek, Wilf, and Zeilberger (Wellesley, Mass.: A. K. Peters, 1996). 

The principal tool for dealing with sums of binomial coefficients in a sys- 
tematic, mechanical way is to exploit the properties of hypergeometric functions, 
which are infinite series defined as follows in terms of rising factorial powers: 


Q1,---; Am GPa 2 
tae Jp, (39) 
An introduction to these important functions can be found in Sections 5.5 and 
5.6 of CMath. See also J. Dutka, Archive for History of Exact Sciences 31 
(1984), 15-34, for historical references. 
The concept of binomial coefficients has several significant generalizations, 
which we should discuss briefly. First, we can consider arbitrary real values of the 


lower index k in G); see exercises 40 through 45. We also have the generalization 


@ Usd eg) ae 
k/4 (=q) -= ae ee ee 
which becomes the ordinary binomial coefficient Gha = (i) when q approaches 


the limiting value 1; this can be seen by dividing each term in numerator and 
denominator by 1 — q. The basic properties of such “gq-nomial coefficients” are 
discussed in exercise 58. 

However, for our purposes the most important generalization is the multi- 
nomial coefficient 


ky + k2 +- + km (ky + ko +: + km)! y 
( ki, kos... km ) hifiuckg oo = (41) 
The chief property of multinomial coefficients is the generalization of Eq. (13): 
wats n Th kı ke kin 
(£1 + ots H Em) 5 on nt p27... OH. (42) 


kitkət e +km=n 


It is important to observe that any multinomial coefficient can be expressed in 
terms of binomial coefficients: 


kitkat +km _ ky tke kı+k2+k3 e (43) 
ky, ko,..-,km ky ki +k ON kit thm 7” : 
so we may apply the techniques that we already know for manipulating binomial 
coefficients. Both sides of Eq. (20) are the trinomial coefficient 


Gare 
k, m—k,r—m/]- 
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Table 2 
STIRLING NUMBERS OF BOTH KINDS 

n n n n nM n n n n 
i A [i] | H A H [e] [7] A 
o| 1 0 0 0 0 0 0 o0 o0 
ıl 0 1 0 0 0 0 0 0 o0 
2! 0 1 1 0 0 0 0 0 o0 
3| 0 2 3 1 0 0 0 0 0 
4| 0 6 11 6 1 0 0 0 0 
5| 0 24 50 35 10 1 0 0 0 
6l 0 120 274 225 85 15 1 0 0 
7| 0 720 1764 1624 735 175 2 1 0 
s| 0 5040 13068 13132 6769 1960 322 28 1 

n n n n n n n n n 
i i {i} {3} ea {i} {5} er erst 
ol 1 0 0 0 0 0 0 o0 o0 
ıl 0 1 0 0 0 0 0 0 o0 
2| 0 1 1 0 0 0 0 0 o0 
3| 0 1 3 1 0 0 0o o0 0 
4| 0 1 7 6 1 0 0 0 0 
5| 0 1 15 25 10 1 0 0 0 
6| 0 1 31 90 65 15 1 0 0 
7| 0 1 63 301 350 140 2 #1 0 
s| 0 1 127 966 1701 1050 266 28 1 


For approximations valid when n is large, see L. Moser and M. Wyman, J. London Math. 
Soc. 33 (1958), 133-146; Duke Math. J. 25 (1958), 29-43; D. E. Barton, F. N. David, and 
M. Merrington, Biometrika 47 (1960), 439-445; 50 (1963), 169-176; N. M. Temme, Studies in 
Applied Math. 89 (1993), 233-243; H. S. Wilf, J. Combinatorial Theory A64 (1993), 344-349; 
H.-K. Hwang, J. Combinatorial Theory A71 (1995), 343-351. 


We conclude this section with a brief analysis of the transformation from 
a polynomial expressed in powers of x to a polynomial expressed in binomial 
coefficients. The coefficients involved in this transformation are called Stirling 
numbers, and these numbers arise in the study of numerous algorithms. 

Stirling numbers come in two flavors: We denote Stirling numbers of the first 
kind by [7'], and those of the second kind by {7}. These notations, due to Jovan 
Karamata [Mathematica (Cluj) 9 (1935), 164-178], have compelling advantages 
over the many other symbolisms that have been tried [see D. E. Knuth, AMM 
99 (1992), 403-422]. We can remember the curly braces in {7°} because curly 
braces denote sets, and ea is the number of ways to partition a set of n elements 
into k disjoint subsets (exercise 64). The other Stirling numbers [}] also have 
a combinatorial interpretation, which we will study in Section 1.3.3: [z] is the 
number of permutations on n letters having k cycles. 

Table 2 displays Stirling’s triangles, which are in some ways analogous to 
Pascal’s triangle. 
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Stirling numbers of the first kind are used to convert from factorial powers 
to ordinary powers: 


For example, from Table 2, 
x x2 1 
({)=-F- Tayo — 102 + 352° — 502 + 242). 


Stirling numbers of the second kind are used to convert from ordinary powers 
to factorial powers: 


seper epe 


This formula was, in fact, Stirling’s original reason for studying the numbers 
{?} in his Methodus Differentialis (London: 1730). From Table 2 we have, for 
example, 


> 1207) +240(77) $ 150(7) +30(5) 4 P: 


We shall now list the most important identities involving Stirling numbers. 
In these equations, the variables m and n always denote nonnegative integers. 


Scere 


eae, 


Inversion formulas (compare with Eq. (33)): 


ETT eae = tm JA H eia Car) 
k k 


Special values: 


Gini Galan & 


Addition formulas: 


(46) 
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mgee Pew rje Crh- 


Expansion formulas: 


abo ea 


Di A y ola! 
ee lea 
ttl ee (55) 


k 


piggt gjet (4th). a 


k<n 


Some other fundamental Stirling number identities appear in exercises 1.2.6-61 
and 1.2.7-6, and in Eqs. (23), (26), (27), and (28) of Section 1.2.9. 

Eq. (49) is just one instance of a general phenomenon: Both kinds of Stirling 
numbers [,,"_,] and {,,”,,} are polynomials in n of degree 2m, whenever m is a 
nonnegative integer. For example, the formulas for m = 2 and m = 3 are 


[nal =G) +207): {aot Cr) eC 
hel OE C Gaag C CE) 


57 
Therefore it makes sense to define the numbers |„”,„] and {,„”,} for arbitrary 
real (or complex) values of r. With this generalization, the two kinds of Stirling 


numbers are united by an interesting duality law 


te leap 8) 
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which was implicit in Stirling’s original discussion. Moreover, Eq. (45) remains 


true 


in general, in the sense that the infinite series 


aoa ae o 


converges whenever the real part of z is positive. The companion formula, 
Eq. (44), generalizes in a similar way to an asymptotic (but not convergent) 


series: E 
zE =D eaa (<1)*2" + O(2"7™71). (60) 
(See exercise 65.) Sections 6.1, 6.2, and 6.5 of CMath contain additional informa- 


tion 


about Stirling numbers and how to manipulate them in formulas. See also 


exercise 4.7—21 for a general family of triangles that includes Stirling numbers 
as a very special case. 


EXE 


RCISES 

00| How many combinations of n things taken n — 1 at a time are possible? 

00] What is (8)? 

00] How many bridge hands (13 cards out of a 52-card deck) are possible? 

10| Give the answer to exercise 3 as a product of prime numbers. 

05] Use Pascal’s triangle to explain the fact that 114 = 14641. 

10] Pascal’s triangle (Table 1) can be extended in all directions by use of the 


addition formula, Eq. (9). Find the three rows that go on top of Table 1 (i.e., for 
r = —1, —2, and —3). 


n 


7. [12] If n is a fixed positive integer, what value of k makes ( a) a maximum? 
8. [00] What property of Pascal’s triangle is reflected in the “symmetry condition,” 
Eq. (6)? 
9. [01] What is the value of (")? (Consider all integers n.) 
> 10. [M25] If p is prime, show that: 
n n 
a = |—| (modulo p). 
a al a 
b) (2) =0 (modulo p), for 1 < & < p=1. 
pally _ k 
c) k = (-1)" (modulo p), for 0 < k < p- 1. 
d) (P$) =0 (modulo p), for 2 < k < p— 1. 


e) (É. Lucas, 1877.) 
(e) = (ities) (emoap) (node x) 


f) If the p-ary number system representations of n and k are 


n = arp” +- + aip + ao, n\ _ (ar a1 \ ( a0 
aes oe then eee cate (modulo p). 
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> 11. [M20] (E. Kummer, 1852.) Let p be prime. Show that if p” divides 
4% + ”) 
a 
but p does not, then n is equal to the number of carries that occur when a is added 
to b in the p-ary number system. [Hint: See exercise 1.2.5-12.] 


n+1 


12. [M22] Are there any positive integers n for which all the nonzero entries in the 
nth row of Pascal’s triangle are odd? If so, find all such n. 


13. [M13] Prove the summation formula, Eq. (10). 
14. [M21] Evaluate Xk o k’. 
15. [M15] Prove the binomial formula, Eq. (13). 


16. [M15] Given that n and k are positive integers, prove the symmetrical identity 


rt") 7 ur"): 
> 17. [M18] Prove the Chu-Vandermonde formula (21) from Eq. (15), using the idea 
that (1+ 2)"t* =(1+2)"(1+2)*. 
18. [M15] Prove Eq. (22) using Eqs. (21) and (6). 
19. [M18] Prove Eq. (23) by induction. 


20. [M20] Prove Eq. (24) by using Eqs. (21) and (19), then show that another use of 
Eq. (19) yields Eq. (25). 


> 21. [M05] Both sides of Eq. (25) are polynomials in s; why isn’t that equation an 
identity in s? 
22. [M20] Prove Eq. (26) for the special case s = n— 1 -r+ nt. 
23. [M13] Assuming that Eq. (26) holds for (r,s,t,n) and (r, s — t, t, n — 1), prove 
it for (r, s +1, t, n). 


24. [M15] Explain why the results of the previous two exercises combine to give a 
proof of Eq. (26). 


25. [HM30| Let the polynomial A,,(x,t) be defined as in Example 4 (see Eq. (30)). 
Let z = tt! — xt. Prove that >, Ax(r,t)z* = x", provided that x is close enough to 1. 
[Note: If t = 0, this result is essentially the binomial theorem, and this equation is an 
important generalization of that theorem. The binomial theorem (15) may be assumed 
in the proof.] Hint: Start with multiples of a special case of (34), 


Dew (ty = 


J 


26. [HM25] Using the assumptions of the previous exercise, prove that 
rt+l1 


D0) 7 yest 


k 


27. [HM21] Solve Example 4 in the text by using the result of exercise 25; and prove 
Eq. (26) from the preceding two exercises. [Hint: See exercise 17.] 
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28. [M25] Prove that 
r+tk\ (s—tk\ _ r+s—k\ k 
D k CÈ n—k ye 
k k>0 
if n is a nonnegative integer. 


29. [M20] Show that Eq. (34) is just a special case of the general identity proved in 
exercise 1.2.3-33. 


30. [M24] Show that there is a better way to solve Example 3 than the way used in 
the text, by manipulating the sum so that Eq. (26) applies. 


31. [M20] Evaluate 
a fal | Gosia pees 
; k n—k m+n 


in terms of r, s, m, and n, given that m and n are integers. Begin by replacing 


co) Blagg) G) 


j 


32. [M20] Show that X ,[}]x" = x”, where x” is the rising factorial power defined in 
Eq. 1.2.5-(19). 

33. [M20] (A. Vandermonde, 1772.) Show that the binomial formula is valid also 
when it involves factorial powers instead of the ordinary powers. In other words, prove 


that 
n n n— na NÝR ack 
ety = (pAr: (+4) =h) i 
k k 


34. [M23] (Torelli’s sum.) In the light of the previous exercise, show that Abel’s 
generalization, Eq. (16), of the binomial formula is true also for rising powers: 


(ety) =D (pele — be + Dy + bey 


k 
35. [M23] Prove the addition formulas (46) for Stirling numbers directly from the 
definitions, Eqs. (44) and (45). 

36. [M10] What is the sum )>,, (%) of the numbers in each row of Pascal’s triangle? 
What is the sum of these numbers with alternating signs, >, (7%) (—1)*? 

37. [M10] From the answers to the preceding exercise, deduce the value of the sum 
of every other entry in a row, (9) + (5) + (7) +- 

38. [HM30] (C. Ramus, 1834.) Generalizing the result of the preceding exercise, show 
that we have the following formula, given that 0 < k < m: 


n n n 4 jn” — j(n— 2k)r 
ee? ma o m dy (2cos 7) m Í 


0<j<m 


ORORO] 


[Hint: Find the right combinations of these coefficients multiplied by mth roots of 
unity.] This identity is particularly remarkable when m > n. 


For example, 
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39. [M10] What is the sum }°,[7] of the numbers in each row of Stirling’s first 
triangle? What is the sum of these numbers with alternating signs? (See exercise 36.) 
40. [HM17| The beta function B(x, y) is defined for positive real numbers x, y by the 
formula B(x, y) = i, t?—-1(1—t)¥ "+ dt. 

a) Show that B(x,1) = B(1,x) = 1/z. 

b) Show that B(x + 1, y) + B(x, y+ 1) = B(x, y). 

c) Show that B(x,y) = ((£ + y)/y) Bla, y + 1). 
41. [HM22] We proved a relation between the gamma function and the beta function 
in exercise 1.2.5-19, by showing that Im(x) = m*B(x,m-+1), if m is a positive integer. 


a) Prove that 


B(x, y) = P EST (x, y+m+ 1) 
b) Show that nana 
SENT WY 
B(x, y) = Ter 


42. [HM10] Express the binomial coefficient (;) in terms of the beta function defined 
above. (This gives us a way to extend the definition to all real values of k.) 


43. [HM20] Show that B(1/2,1/2) = m. (From exercise 41 we may now conclude that 
I(1/2) = vr.) 


44. [HM20] Using the generalized binomial coefficient suggested in exercise 42, show 


that 
(ia) =?" G) 
1/2 r 
45. [HM21] Using the generalized binomial coefficient suggested in exercise 42, find 
limy— oo (o) /r*. 
46. [M21] Using Stirling’s approximation, Eq. 1.2.5-(7), find an approximate value 
of (F); assuming that both x and y are large. In particular, find the approximate 


size of (2) when n is large. 
47. [M21] Given that k is an integer, show that 


(1) ("~ =! _ (77) Ga. 4 P ‘ee 4k 
k k Ak k ~ \Qk k : 
Give a simpler formula for the special case r = —1/2. 

48. [M25] Show that 


n\ CD _ n! ... iil 
>) k+ s(x +1)... (x+n) PTOA 


k>0 n 
if the denominators are not zero. [Note that this formula gives us the reciprocal of a 
binomial coefficient, as well as the partial fraction expansion of 1/z(x + 1)... (x+ n).] 
49. [M20] Show that the identity (1 + x)” = (1 — z°)" (1 — z)” implies a relation on 
binomial coefficients. 
50. [M20] Prove Abel’s formula, Eq. (16), in the special case x + y = 0. 


51. [M21] Prove Abel’s formula, Eq. (16), by writing y = (x + y) — x, expanding the 
right-hand side in powers of (x + y), and applying the result of the previous exercise. 
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52. [HM11] Prove that Abel’s binomial formula (16) is not always valid when n is not 
a nonnegative integer, by evaluating the right-hand side when n = x ly=z=1. 


53. [M25] (a) Prove the following identity by induction on m, where m and n are 
integers: 


SCE Jer (r+s)k)=(m+4+1)\(n i) nee | aan 


k=0 


(b) Making use of important relations from exercise 47, 


(=°) _ (-1)" ~ (712) — (-1)""1 (2) — (=t a 5 

n J 2a ln’ n 22n(2n—1)\n 22n-1(2n— 1) n mm 
show that the following formula can be obtained as a special case of the identity in 
part (a): 


So armas > i n Oe ee) 
a Ok n—-k /2k-1 2n m n-m 2n 
(This result is considerably more general than Eq. (26) in the case r = —1, s = 0, 


t= -—2.) 


54. [M21] Consider Pascal’s triangle (as shown in Table 1) as a matrix. What is the 
inverse of that matrix? 


55. [M21] Considering each of Stirling’s triangles (Table 2) as matrices, determine 
their inverses. 


56. [20] (The combinatorial number system.) For each integer n = 0, 1, 2, ..., 20, 


find three integers a, b, c for which n = (3) + (2) + ({) anda >b > c> 0. Can you see 


how this pattern can be continued for higher values of n? 


57. [M22] Show that the coefficient am in Stirling’s attempt at generalizing the fac- 
torial function, Eq. 1.2.5—(12), is 


Sar Ea) ne 


k>1 


58. [M23] (H. A. Rothe, 1811.) In the notation of Eq. (40), prove the “gq-nomial 
theorem”: 


k hq 


Also find g-nomial generalizations of the fundamental identities (17) and (21). 


59. [M25] A sequence of numbers Ank, n > 0, k > 0, satisfies the relations Ano = 1, 
Aok = Ook, Ank = A(n—1)k + A(n—1)(k—-1) + Cj for nk > 0. Find Ank. 


60. [M23] We have seen that (7) is the number of combinations of n things, k at a 
time, namely the number of ways to choose k different things out of a set of n. The 
combinations with repetitions are similar to ordinary combinations, except that we 
may choose each object any number of times. Thus, the list (1) would be extended to 
include also aaa, aab, aac, aad, aae, abb, etc., if we were considering combinations with 
repetition. How many k-combinations of n objects are there, if repetition is allowed? 
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61. [M25] Evaluate the sum 


thereby obtaining a companion formula for Eq. (55). 


62. [M23] The text gives formulas for sums involving a product of two binomial 
coefficients. Of the sums involving a product of three binomial coefficients, the following 
one and the identity of exercise 31 seem to be most useful: 


eee gia nee = Kimra integer l, m,n > 0. 


l+k m+k/\n+k i'm! n! 


(The sum includes both positive and negative values of k.) Prove this identity. 
[Hint: There is a very short proof, which begins by applying the result of exercise 31.] 
63. [M30] Ifl, m, and n are integers and n > 0, prove that 


GC aes oe ae 


j,k 


64. [M20] Show that {> } is the number of ways to partition a set of n elements 
into m nonempty disjoint subsets. For example, the set {1,2,3,4} can be partitioned 
into two subsets in {5} = 7 ways: {1,2,3}{4}; {1,2,4} {3}; {1,3,4}{2}; {2,3,4H1}; 
{1,2}{3, 4}; {1,3 H2, 4}; {1,4}{2,3}. Hint: Use Eq. (46). 


65. [HM35] (B. F. Logan.) Prove Eqs. (59) and (60). 
66. [HM30] Suppose x, y, and z are real numbers satisfying 


n n n—1 
where x > n— 1, y > n—1, z >n -—2, and n is an integer > 2. Prove that 
x y z í ; 
< + ae 
eae a Cog) if and only if y > z; 


z y z ; : 
< 4 <z. 
ta) < an (7) if and only if yz 


67. [M20] We often need to know that binomial coefficients aren’t too large. Prove 
the easy-to-remember upper bound 


k 
n ne 
< (| — >k>0. 
Jah whenn>k>0 


68. [M25] (A. de Moivre.) Prove that, if n is a nonnegative integer, 


E (g) a-p" - npl = 2f ( rnp pela = prH, 


k 
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1.2.7. Harmonic Numbers 


The following sum will be of great importance in our later work: 


1 1 1 Gl 
melk to ttt =) a: n= 0. (1) 
k=1 

This sum does not occur very frequently in classical mathematics, and there is 
no standard notation for it; but in the analysis of algorithms it pops up nearly 
every time we turn around, and we will consistently call it H,. Besides Hp, 
the notations hn and Sn and w(n+1)+7 are found in mathematical literature. 
The letter H stands for “harmonic,” and we speak of H, as a harmonic number 
because (1) is customarily called the harmonic series. Chinese bamboo strips 
written before 186 B.C. already explained how to compute Hy 9 = 7381/2520, as 
an exercise in arithmetic. [See C. Cullen, Historia Math. 34 (2007), 10—44.] 

It may seem at first that H, does not get too large when n has a large value, 
since we are always adding smaller and smaller numbers. But actually it is not 
hard to see that H, will get as large as we please if we take n to be big enough, 
because 


Hom > 1+ S (2) 
This lower bound follows from the observation that, for m > 0, we have 
1 1 1 
ard 2 am +1 2m +4 2 + 2m+1 
1 1 
> Agm 4 ape = Hm + 5. 


toma + gmt T° 1 pat 


So as m increases by 1, the left-hand side of (2) increases by at least 5. 

It is important to have more detailed information about the value of Hn 
than is given in Eq. (2). The approximate size of H,, is a well-known quantity 
(at least in mathematical circles) that may be expressed as follows: 


1 1 1 
Ay, =lnn+y+ 


1 
mn Tan? me © «= © < a5 ant” (3) 


Here y = 0.5772156649... is Fuler’s constant, introduced by Leonhard Euler in 
Commentarii Acad. Sci. Imp. Pet. 7 (1734), 150-161. Exact values of H,, for 
small n, and a 40-place value for y, are given in the tables in Appendix A. We 
shall derive Eq. (3) in Section 1.2.11.2. 

Thus H,, is reasonably close to the natural logarithm of n. Exercise 7(a) 
demonstrates in a simple way that H,, has a somewhat logarithmic behavior. 

In a sense, H,, just barely goes to infinity as n gets large, because the similar 
sum 

lhe ee (4) 


stays bounded for all n, when r is any real-valued exponent greater than unity. 
(See exercise 3.) We denote the sum in Eq. (4) by H”. 
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When the exponent r in Eq. (4) is at least 2, the value of HÊ? is fairly close 
to its maximum value HX’, except for very small n. The quantity HZ’ is very 
well known in mathematics as Riemann’s zeta function: 


HO =r) =e 6) 
k>1 


If r is an even integer, the value of ¢(r) is known to be equal to 


s i Qn)" 
HY) = 5 |B; ( al , integer r/2 > 1, (6) 


where B, is a Bernoulli number (see Section 1.2.11.2 and Appendix A). In 
particular, 


4 
HƏ- HƏ- wo pD- (7) 
E 6’ = 90’ i 945 iè 9450 
These results are due to Euler; for discussion and proof, see CMath, §6.5. 
Now we will consider a few important sums that involve harmonic numbers. 
First, 


5 E E A (8) 


This follows from a simple interchange of summation: 


Formula (8) is a special case of the sum X}; (E ) Hy, which we will now 
determine using an important technique called summation by parts (see exer- 
cise 10). Summation by parts is a useful way to evaluate ` a,b, whenever the 
quantities ` a, and (by, 41 — bp) have simple forms. We observe in this case that 


(= (nat) (nea) 
m) \m+l1 m+1/’ 
and therefore 
k k+1 1 k 
(7) Be = (0 1) Em- ga) ~ on 


($)m= er 


iL 


= TE a 148) 


=0 
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Applying Eq. 1.2.6—(11) yields the desired formula: 


(1) (a) oe pea) 


k=1 


(This derivation and its final result are analogous to the evaluation of 


ie “i P 7 n™t1 ( 1 )+ 1 
: T nr T EE nn m41 (m + 1)? 


using what calculus books call integration by parts.) 
We conclude this section by considering a different kind of sum, 5°, atk, Rs 
which we will temporarily denote by Sn for brevity. We find that 


s p(Q Gi Asp) 


Hence Sp41 = (x£ + 1)Sn + ((x +1)?! — 1)/(n + 1), and we have 
Sri Sn 1 1 


(29+1)Pt!  (@+1)" n+1 (n++1)(z+1)+1' 


This equation, together with the fact that Sı = x, shows us that 


Sn ” 
jane Pe geek G0) 


The new sum is part of the infinite series 1.2.9-(17) for n(1/(1—1/(«+1))) = 
In(1 + 1/2), and when x > 0, the series is convergent; the difference is 


1 1 1 1 
2 ka +I ~ mnt DGE > +E (nter 


This proves the following theorem: 
Theorem A. If x > 0, then 


(Pim = (e+)? (1, In(14 1) bse 


k=1 


where 0 < e < 1/(x(n+1)). 1 


EXERCISES 
1. [01] What are Ho, Hı, and H2? 


2. [13] Show that the simple argument used in the text to prove that Həm > 1+m/2 
can be slightly modified to prove that Həm < 1 + m. 
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3. [M21] Generalize the argument used in the previous exercise to show that, for 
r > 1, the sum HẸ? remains bounded for all n. Find an upper bound. 


> 4. [10] Decide which of the following statements are true for all positive integers n: 
(a) Hn <Inn. (b) Hn > Inn. (c) Hn > nn+7. 


5. [15] Give the value of Hı10000 to 15 decimal places, using the tables in Appendix A. 


6. [M15] Prove that the harmonic numbers are directly related to Stirling’s numbers, 
which were introduced in the previous section; in fact, 


t= [Eo 


7. [M21] Let T(m,n) = Hm + An — Hmn. (a) Show that when m or n increases, 
T(m,n) never increases (assuming that m and n are positive). (b) Compute the 
minimum and maximum values of T(m, n) for m,n > 0. 


8. [HM18] Compare Eq. (8) with XZ}; nk; estimate the difference as a function 
of n. 


> 9. [M18] Theorem A applies only when x > 0; what is the value of the sum consid- 
ered when x = —1? 


10. [M20] (Summation by parts.) We have used special cases of the general method 
of summation by parts in exercise 1.2.4—42 and in the derivation of Eq. (9). Prove the 
general formula 


5 (ak+1 — ak) bk = anbn — arbi 5 ak+ı(bk+1 — bk). 


1<k<n 1<k<n 


> 11. [M21] Using summation by parts, evaluate 


1 
een 
1<k<n k(k — 1) 
> 12. [M10] Evaluate HQ) correct to at least 100 decimal places. 
13. [M22] Prove the identity 


n 


2a aro 


k=1 


(Note in particular the special case x = 0, which gives us an identity related to exercise 
1.2.6-48.) 
14. [M22] Show that 5? , H/k = 4(H2 + HY), and evaluate P_, Hy /(k + 1). 
> 15. [M23] Express )77_, HZ in terms of n and Hn. 
16. [18] Express the sum 1 + a ++ z in terms of harmonic numbers. 
17. [M24] (E. Waring, 1782.) Let p be an odd prime. Show that the numerator of 
Hy~-1 is divisible by p. 
18. [M33] (J. Selfridge.) What is the highest power of 2 that divides the numerator 
ofl+i+ tz? 


> 19. [M30] List all nonnegative integers n for which Hn is an integer. [Hint: If Hn 
has odd numerator and even denominator, it cannot be an integer.] 


1.2.8 FIBONACCI NUMBERS 79 


20. [HM22] There is an analytic way to approach summation problems such as the one 
leading to Theorem A in this section: If f(z) = 7,59 axx", and this series converges 
for x = Xo, prove that i 


1 

> anak Hy _ f ao f(zoy) dy. 

k>0 0 =y 

21. [M24] Evaluate $% Hk/(n+1-— k). 

22. [M28] Evaluate J` }-o Hk Hn-k. 

23. [HM20] By considering the function I”(x)/I (x), generalize H, to noninteger 
values of n. You may use the fact that T”(1) = —y, anticipating the next exercise. 
24. [HM21] Show that 


oT (4 eo") = hy 


k>1 

(Consider the partial products of this infinite product.) 

25. [M21] Let HS” = X a<j<r<n 1/(J"k”). What are HO”) and HEV”? Prove the 
general identity HE” 4 HO” = HO HO pte) 


1.2.8. Fibonacci Numbers 


The sequence 
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ..., (1) 


in which each number is the sum of the preceding two, plays an important role 
in at least a dozen seemingly unrelated algorithms that we will study later. The 
numbers in the sequence are denoted by Fn, and we formally define them as 


Fo = 0; Fi, = 1; Frye = Frait Fn, n= 0. (2) 


This famous sequence was published in 1202 by Leonardo Pisano (Leonardo 
of Pisa), who is sometimes called Leonardo Fibonacci (Filius Bonaccti, son of 
Bonaccio). His Liber Abaci (Book of the Abacus) contains the following exercise: 
“How many pairs of rabbits can be produced from a single pair in a year’s time?” 
To solve this problem, we are told to assume that each pair produces a new pair 
of offspring every month, and that each new pair becomes fertile at the age of 
one month. Furthermore, the rabbits never die. After one month there will be 
2 pairs of rabbits; after two months, there will be 3; the following month the 
original pair and the pair born during the first month will both usher in a new 
pair and there will be 5 in all; and so on. 

Fibonacci was by far the greatest European mathematician of the Middle 
Ages. He studied the work of al-Khwarizmi (after whom “algorithm” is named, 
see Section 1.1) and he added numerous original contributions to arithmetic and 
geometry. The writings of Fibonacci were reprinted in 1857 [B. Boncompagni, 
Scritti di Leonardo Pisano (Rome, 1857-1862), 2 vols.; Fn appears in Vol. 1, 283- 
285]. His rabbit problem was, of course, not posed as a practical application to 
biology and the population explosion; it was an exercise in addition. In fact, 
it still makes a rather good computer exercise about addition (see exercise 3); 
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Fibonacci wrote: “It is possible to do [the addition] in this order for an infinite 
number of months.” 

Before Fibonacci wrote his work, the sequence (F;,) had already been dis- 
cussed by Indian scholars, who had long been interested in rhythmic patterns 
that are formed from one-beat and two-beat notes or syllables. The number of 
such rhythms having n beats altogether is F,,41; therefore both Gopala (before 
1135) and Hemacandra (c. 1150) mentioned the numbers 1, 2, 3, 5, 8, 13, 21, 
34, ... explicitly. [See P. Singh, Historia Math. 12 (1985), 229-244; see also 
exercise 4.5.3-32.] 

The same sequence also appears in the work of Johannes Kepler, 1611, who 
was musing about the numbers he saw around him [J. Kepler, The Six-Cornered 
Snowflake (Oxford: Clarendon Press, 1966), 21]. Kepler was presumably unaware 
of Fibonacci’s brief mention of the sequence. Fibonacci numbers have often been 
observed in nature, probably for reasons similar to the original assumptions of 
the rabbit problem. [See Conway and Guy, The Book of Numbers (New York: 
Copernicus, 1996), 113-126, for an especially lucid explanation.| 

A first indication of the intimate connections between F, and algorithms 
came to light in 1837, when E. Léger used Fibonacci’s sequence to study the 
efficiency of Euclid’s algorithm. He observed that if the numbers m and n in 
Algorithm 1.1E are not greater than Fk, step E2 will be executed at most k — 1 
times. This was the first practical application of Fibonacci’s sequence. (See 
Theorem 4.5.3F.) During the 1870s the mathematician É. Lucas obtained very 
profound results about the Fibonacci numbers, and in particular he used them to 
prove that the 39-digit number 21?7—1 is prime. Lucas gave the name “Fibonacci 
numbers” to the sequence (Fn), and that name has been used ever since. 

We already have examined the Fibonacci sequence briefly in Section 1.2.1 
(Eq. (3) and exercise 4), where we found that ¢"~? < Fa < ¢"~1 if n is a positive 
integer and if 


o=3(1+Vv5). (3) 


We will see shortly that this quantity, ¢, is intimately connected with the 
Fibonacci numbers. 

The number ¢ itself has a very interesting history. Euclid called it the 
“extreme and mean ratio”; the ratio of A to B is the ratio of A+ B to A, if the 
ratio of A to B is ¢. Renaissance writers called it the “divine proportion”; and in 
the last century it has commonly been called the “golden ratio.” Many artists and 
writers have said that the ratio of ¢ to 1 is the most aesthetically pleasing propor- 
tion, and their opinion is confirmed from the standpoint of computer program- 
ming aesthetics as well. For the story of ¢, see the excellent article “The Golden 
Section, Phyllotaxis, and Wythoff’s Game,” by H. S. M. Coxeter, Scripta Math. 
19 (1953), 135-143; see also Chapter 8 of The 2nd Scientific American Book of 
Mathematical Puzzles and Diversions, by Martin Gardner (New York: Simon 
and Schuster, 1961). Several popular myths about ¢ have been debunked by 
George Markowsky in College Math. J. 23 (1992), 2-19. The fact that the ratio 
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Fy41/Fn approaches ¢ was known to the early European reckoning master Simon 
Jacob, who died in 1564 [see P. Schreiber, Historia Math. 22 (1995), 422—424]. 

The notations we are using in this section are a little undignified. In much 
of the sophisticated mathematical literature, Fn is called u, instead, and ¢ is 
called 7. Our notations are almost universally used in recreational mathematics 
(and some crank literature!) and they are rapidly coming into wider use. The 
designation ¢ comes from the name of the Greek artist Phidias who is said to have 
used the golden ratio in his sculpture. [See T. A. Cook, The Curves of Life (1914), 
420.] The notation F» is in accordance with that used in the Fibonacci Quarterly, 
where the reader may find numerous facts about the Fibonacci sequence. A good 
reference to the classical literature about F» is Chapter 17 of L. E. Dickson’s 
History of the Theory of Numbers 1 (Carnegie Inst. of Washington, 1919). 

The Fibonacci numbers satisfy many interesting identities, some of which 
appear in the exercises at the end of this section. One of the most commonly 
discovered relations, mentioned by Kepler in a letter he wrote in 1608 but first 
published by J. D. Cassini [Histoire Acad. Roy. Paris 1 (1680), 201], is 


Fn+iFn-1 — F? =(-1)", (4) 


which is easily proved by induction. A more esoteric way to prove the same 
formula starts with a simple inductive proof of the matrix identity 


Pa Be. /L 1Y 
( F, m ~ ` a) l (5) 
We can then take the determinant of both sides of this equation. 


Relation (4) shows that F, and Fp+1 are relatively prime, since any common 
n 


divisor would have to be a divisor of (—1)”. 
From the definition (2) we find immediately that 

Fn43 = Fy+2 + Fn4+1 = 2Fn+1 ai Fn; Fn44 = 3Fn+1 + 2F,; 

and, in general, by induction that 


Fnim = Inf, iF Fm-1Fn (6) 


for any positive integer m. 
If we take m to be a multiple of n in Eq. (6), we find inductively that 


Fnk is a multiple of Fy. 


Thus every third number is even, every fourth number is a multiple of 3, every 
fifth is a multiple of 5, and so on. 

In fact, much more than this is true. If we write gcd(m,n) to stand for the 
greatest common divisor of m and n, a rather surprising theorem emerges: 


Theorem A (É. Lucas, 1876). A number divides both F,, and F, if and only 
if it is a divisor of Fy, where d = gcd(m, n); in particular, 


gced(Fin, Fa) = Fgca(m,n): (7) 
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Proof. This result is proved by using Euclid’s algorithm. We observe that 
because of Eq. (6) any common divisor of Fm and F, is also a divisor of Frm; 
and, conversely, any common divisor of Fn+m and Fn is a divisor of FmFn+1- 
Since Fn+1 is relatively prime to Fn, a common divisor of F,4m and Fn also 
divides Fm. Thus we have proved that, for any number d, 


d divides Fm and Fn if and only if d divides Fm+n and Fy. (8) 


We will now show that any sequence (Fn) for which statement (8) holds, and for 
which Fo = 0, satisfies Theorem A. 

First it is clear that statement (8) may be extended by induction on k to 
the rule 


d divides Fm and Fn if and only if d divides Fm+kn and Fn, 
where k is any nonnegative integer. This result may be stated more succinctly: 
d divides Fm moan and Fn if and only if d divides Fm and Fy. (9) 


Now if r is the remainder after division of m by n, that is, if r = m mod n, 
then the common divisors of {Fm, Fn} are the common divisors of { Fn, Fp}. It 
follows that throughout the manipulations of Algorithm 1.1E the set of common 
divisors of { Fm, Fn} remains unchanged as m and n change; finally, when r = 0, 
the common divisors are simply the divisors of Fo = 0 and Fyca(mjn)- d 

Most of the important results involving Fibonacci numbers can be deduced 
from the representation of F, in terms of ¢, which we now proceed to derive. 
The method we shall use in the following derivation is extremely important, and 
the mathematically oriented reader should study it carefully; we will study the 
same method in detail in the next section. 

We start by setting up the infinite series 


G(z) = Fo + Fiz + Foz? + Faz? + Fuz*+--- 
S24 27 +229 + 3244-.-, (10) 
We have no a priori reason to expect that this infinite sum exists or that the 
function G(z) is at all interesting—but let us be optimistic and see what we 
can conclude about the function G(z) if it does exist. The advantage of such a 
procedure is that G(z) is a single quantity that represents the entire Fibonacci 
sequence at once; and if we find out that G(z) is a “known” function, its 
coefficients can be determined. We call G(z) the generating function for the 
sequence (Fn). 
We can now proceed to investigate G(z) as follows: 
2G(z) = Foz + F12? + Foz? + Fzt ts, 
2 G2) = Foz? + Fuze + Foz +; 
by subtraction, therefore, 
(1 — z- 2°)G(z2) = Fo + (Fi — Fo)z + (Fo — Fi — Fo)z? 
+ (F; — F — Fi)? + (Fa — F; — Fo)zf +. 
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All terms but the second vanish because of the definition of F;,, so this expression 
equals z. Therefore we see that, if G(z) exists, 


G(z) =2/(-—z-2*). (11) 


In fact, this function can be expanded in an infinite series in z (a Taylor series); 
working backwards we find that the coefficients of the power series expansion of 
Eq. (11) must be the Fibonacci numbers. 

We can now manipulate G(z) and find out more about the Fibonacci se- 
quence. The denominator 1 — z — z? is a quadratic polynomial with the two 
roots $(—1 + V5); after a little calculation we find that G(z) can be expanded 
by the method of partial fractions into the form 


1 1 1 
a =- (1 5): (12) 

where 2 
ĝ=1-¢= }(1- v5). (13) 


The quantity 1/(1 — $z) is the sum of the infinite geometric series 1 + z + 
$727 +--+, so we have 


1 
G(z) = Lap z+g?z? pee 1 a eee 
(2) Z. gz+¢ pz- F ) 
We now look at the coefficient of z”, which must be equal to Fn; hence 
1 
Fa = — (¢" — $”). 1 
z0") (24) 


This is an important closed form expression for the Fibonacci numbers, first 
discovered early in the eighteenth century. (See D. Bernoulli, Comment. Acad. 
Sci. Petrop. 3 (1728), 85-100, §7; see also A. de Moivre, Philos. Trans. 32 (1722), 
162-178, who showed how to solve general linear recurrences in essentially the 
way we have derived (14).) 

We could have merely stated Eq. (14) and proved it by induction. However, 
the point of the rather long derivation above was to show how it would be 
possible to discover the equation in the first place, using the important method 
of generating functions, which is a valuable technique for solving a wide variety 
of problems. 

Many things can be proved from Eq. (14). First we observe that ¢ is a 
negative number (—0.61803...) whose magnitude is less than unity, so @” gets 
very small as n gets large. In fact, the quantity @"/\/5 is always small enough 
so that we have 


F, = ¢"/V5 rounded to the nearest integer. (15) 


Other results can be obtained directly from G(z); for example, 


2_ 1 1 1 2 
G(z) 5 Gee: ' (1 — $2)? m) (16) 
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and the coefficient of 2” in G(z)? is $X ;.—o FkFn-x. We deduce therefore that 


Yo FaFn—e = 3((n+1)(6" + 8") — Fos) 
k=0 
g z((n H 1) (Fn i 2Fn—1) = 2Fn+1) 


= (n — 1)Fn + 2nFy-1. (17) 


(The second step in this derivation follows from the result of exercise 11.) 


EXE 
1. 
pairs 
> 2. 


RCISES 
10| What is the answer to Leonardo Fibonacci’s original problem: How many 
of rabbits are present after a year? 


20] In view of Eq. (15), what is the approximate value of Fioo0? (Use logarithms 


found in Appendix A.) 


3. 


25| Write a computer program that calculates and prints Fı through Fiooo in 


decimal notation. (The previous exercise determines the size of numbers that must be 


hand 
> A, 
5. 
6. 
> T: 


led.) 
14| Find all n for which Fn = n. 
20] Find all n for which F, = n?. 
HM10] Prove Eq. (5). 


15] If n is not a prime number, F» is not a prime number (with one exception). 


Prove this and find the exception. 


15| In many cases it is convenient to define Fn for negative n, by assuming that 
= F,41+ Fn for all integers n. Explore this possibility: What is F_;? What is 


? Can F_n be expressed in a simple way in terms of Fn? 


M20] Using the conventions of exercise 8, determine whether Eqs. (4), (6), (14), 
15) still hold when the subscripts are allowed to be any integers. 


15] Is v5 greater than Fn or less than Fn? 
M20] Show that ¢” = Fao + Fn-1 and 6” = Fa Ê + Fn-1, for all integers n. 


Expr 
> 13. 
T, 8, 


M26] The “second order” Fibonacci sequence is defined by the rule 
Fo = 0, Fy = 1, Fn42 = Fn+1 + Fn + Fn. 


ess Fn in terms of Fn and Fn+1. [Hint: Use generating functions.] 


[M22] Express the following sequences in terms of the Fibonacci numbers, when 
and c are given constants: 


a) ao =T, a1 = S; An+2 = an+1 + an, for n > 0. 
b) bo = 0, bi = 1; bn42 = bn41+bn +c, for n > 0. 


14. 


15. 


[M28] Let m be a fixed positive integer. Find an, given that 
ao=0, a =l; An42 = an+1 + an + (Z), for n > 0. 
[M22] Let f(n) and g(n) be arbitrary functions, and for n > 0 let 


ao=0, ar =l, an42 = an+1 + an + f(n); 
bo=0, bi=1, bay = bn41 + bn + g(n); 
Co = 0, Cı = 1, Cn+2 = Cn4+1 Cn xf (n) H yg(n). 


Express cn in terms of x, yY, an, bn, and Fn. 
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> 16. [M20] Fibonacci numbers appear implicitly in Pascal’s triangle if it is viewed from 
the right angle. Show that the following sum of binomial coefficients is a Fibonacci 
number: n 
k=0 f 
17. [M24] Using the conventions of exercise 8, prove the following generalization of 
Eq. (4): Fn4kFm-k = Fy, Fim = (-1)" Pin-n—KF rr. 
18. [20] Is Fe + Fay always a Fibonacci number? 
> 19. [M27] What is cos 36°? 
20. [M16] Express $ %—o Fe in terms of Fibonacci numbers. 
21. [M25] What is Pko Fex*? 
> 22. [M20] Show that an (7) Fm+k is a Fibonacci number. 
23. [M23] Generalizing the preceding exercise, show that >, (2) FFF En ap is 
always a Fibonacci number. 
24. [HM20] Evaluate the n x n determinant 


1 —1 0 0 ... 00 0 
1 1 —1 0.. 0 0 0 
0 1 1 —1 ... 0 0 0 
0 0 0 0 1 1 —1 
0 0 0 0 0 1 1 
25. [M21] Show that ET 
2# F, =2 k—1 2. 
È (i) 


k odd 
> 26. [M20] Using the previous exercise, show that Fp = 5(P-1)/2 (modulo p) if p is an 
odd prime. 


27. [M20] Using the previous exercise, show that if p is a prime different from 5, then 
either Fp_1 or Fp+ı (not both) is a multiple of p. 


28. [M21] What is Fr41 — oFn? 
> 29. [M23] (Fibonomial coefficients.) Edouard Lucas defined the quantities 


i _ Fy Fp-1..-Fr—k+1 _ if (=) 
kp Fa HUN F 


in a manner analogous to binomial coefficients. (a) Make a table of (i forO<k< 
n < 6. (b) Show that (p) is always an integer because we have 


n n—1 n—1 
(p) =z- k Jenal i 


> 30. [M38] (D. Jarden, T. Motzkin.) The sequence of mth powers of Fibonacci num- 
bers satisfies a recurrence relation in which each term depends on the preceding m + 1 
terms. Show that 


ee (1) -9/1 pm] _ 9, ifm >0. 
k i 


For example, when m = 3 we get the identity F2 — 2F2 H1 2F2 124 F2 13 = 0. 
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31. [M20] Show that Fo,¢@mod1=1—¢-?" and Fon4i¢ mod 1 = ¢-?""14, 


32. [M24] The remainder of one Fibonacci number divided by another is + a Fibo- 
nacci number: Show that, modulo Fn, 


Fr; if m mod 4 = 0; 

F _ (=1)" Fae, if m mod 4 = 1; 
mnir = X (1) F, if m mod 4 = 2; 
(-1)"**" Fa-r, if mmod4= 3. 


33. [HM24] Given that z = 7/2 + iln 4, show that sin nz/sin z = i7” Fn. 

34. [M24] (The Fibonacci number system.) Let the notation k > m mean that k > 
m+2. Show that every positive integer n has a unique representation n = Fx, + Fko + 
-+ + Fkn, where ki > k2 >- > kr > 0. 

35. [M24] (A phi number system.) Consider real numbers written with the digits 0 
and 1 using base ¢; thus (100.1), = ¢?+ ¢~'. Show that there are infinitely many 
ways to represent the number 1; for example, 1 = (.11)4 = (.011111...),. But if 
we require that no two adjacent 1s occur and that the representation does not end 
with the infinite sequence 01010101..., then every nonnegative number has a unique 
representation. What are the representations of integers? 

36. [M32] (Fibonacci strings.) Let Sı = “a”, S2 = “b”, and Sn42 = Sn4iSn, n > 0; 
in other words, S;,42 is formed by placing Sn at the right of S,41. We have S3 = “ba”, 
S4 = “bab”, Ss = “babba”, etc. Clearly Sn has Fn letters. Explore the properties of 
Sn. (Where do double letters occur? Can you predict the value of the kth letter of Sn? 
What is the density of the b’s? And so on.) 


37. [M35] (R. E. Gaskell, M. J. Whinihan.) Two players compete in the following 
game: There is a pile containing n chips; the first player removes any number of chips 
except that he cannot take the whole pile. From then on, the players alternate moves, 
each person removing one or more chips but not more than twice as many chips as the 
preceding player has taken. The player who removes the last chip wins. (For example, 
suppose that n = 11; player A removes 3 chips; player B may remove up to 6 chips, 
and he takes 1. There remain 7 chips; player A may take 1 or 2 chips, and he takes 2; 
player B may remove up to 4, and he picks up 1. There remain 4 chips; player A now 
takes 1; player B must take at least one chip and player A wins in the following turn.) 

What is the best move for the first player to make if there are initially 1000 chips? 


38. [35] Write a computer program that plays the game described in the previous 
exercise and that plays optimally. 


39. [M24] Find a closed form expression for an, given that a9 = 0, a1 = 1, and 
Qn+2 = Qn+1 + 6an for n > 0. 
40. [M25] Solve the recurrence 
fHY=0; f(n)= omin max(1+ f(k),2+f(n—k)), forn>1. 
<k<n 


41. [M25] (Yuri Matiyasevich, 1990.) Let f(x) = |x + '|. Prove that if n = 
Fy, +--+ Fp, is the representation of n in the Fibonacci number system of exercise 34, 
then Fkı+1 +-+- + Fr,+41 = f(én). Find a similar formula for Fh,-1 +--+ + Fr,.-1. 
42. [M26] (D. A. Klarner.) Show that if m and n are nonnegative integers, there is a 
unique sequence of indices kı > k2 >>--- >> kr such that 


m= Fe, + Fro +- + Fr, n = Fkipi + Fro4i +--+ + Fenti- 


(See exercise 34. The k’s may be negative, and r may be zero.) 
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1.2.9. Generating Functions 


Whenever we want to obtain information about a sequence of numbers (an) = 


do, @1,@2,..., we can set up an infinite sum in terms of a “parameter” z, 
G(z) = ao + a1z + a22? + = ) One (1) 
n>0 


We can then try to obtain information about the function G. This function is a 
single quantity that represents the whole sequence; if the sequence (an) has been 
defined inductively (that is, if a, has been defined in terms of ao, a1,...,a@n—1) 
this is an important advantage. Furthermore, we can recover the individual 
values of ag, a1, ... from the function G(z), assuming that the infinite sum in 
Eq. (1) exists for some nonzero value of z, by using techniques of differential 
calculus. 

We call G(z) the generating function for the sequence ao, a1, az, .... The 
use of generating functions opens up a whole new range of techniques, and 
it broadly increases our capacity for problem solving. As mentioned in the 
previous section, A. de Moivre introduced generating functions in order to solve 
the general linear recurrence problem. De Moivre’s theory was extended to 
slightly more complicated recurrences by James Stirling, who showed how to 
apply differentiation and integration as well as arithmetic operations [Methodus 
Differentialis (London: 1730), Proposition 15]. A few years later, L. Euler began 
to use generating functions in several new ways, for example in his papers on 
partitions [Commentarii Acad. Sci. Pet. 13 (1741), 64-93; Novi Comment. Acad. 
Sci. Pet. 3 (1750), 125-169]. Pierre S. Laplace developed the techniques further 
in his classic work Théorie Analytique des Probabilités (Paris: 1812). 

The question of convergence of the infinite sum (1) is of some importance. 
Any textbook about the theory of infinite series will prove that: 


a) If the series converges for a particular value of z = zo, then it converges for 
all values of z with |z| < |Zo|- 


b) The series converges for some z # 0 if and only if the sequence ({/|a;,|) is 
bounded. (If this condition is not satisfied, we may be able to get a conver- 
gent series for the sequence (a,,/n!) or for some other related sequence.) 


On the other hand, it often does not pay to worry about convergence of 
the series when we work with generating functions, since we are only exploring 
possible approaches to the solution of some problem. When we discover the 
solution by any means, however sloppy, we may be able to justify the solu- 
tion independently. For example, in the previous section we used a generating 
function to deduce Eq. (14); yet once such an equation has been found, it is a 
simple matter to prove it by induction, and we need not even mention that we 
used generating functions to discover it. Furthermore one can show that most 
(if not all) of the operations we do with generating functions can be rigorously 
justified without regard to the convergence of the series. See, for example, E. T. 
Bell, Trans. Amer. Math. Soc. 25 (1923), 135-154; Ivan Niven, AMM 76 (1969), 
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871-889; Peter Henrici, Applied and Computational Complex Analysis 1 (Wiley, 
1974), Chapter 1. 
Let us now study the principal techniques used with generating functions. 


A. Addition. If G(z) is the generating function for (an) = ao,ai1,... and 
H(z) is the generating function for (bn) = bo, b1,..., then aG(z) + GH(z) is the 
generating function for (aan + bby) = aao + Bbo, aay + Bbi, ...: 


a 5 Anz" + B 5 baz” = S (aan + bbn)”. (2) 
n>0 n>0 n>0 


B. Shifting. If G(z) is the generating function for (an) = ao,a1,... then 


z™G(z) is the generating function for (an-m) =0,...,0,@0,@1,-.-: 
gee 5 anz” = 5 Qe =e (3) 
n>0 n>m 


The last summation may be extended over all n > 0 if we regard a, = 0 for any 
negative value of n. 

Similarly, (G(z) — ao — a1z—-+++—a@m-_12""~")/z”™ is the generating function 
for (Anim) = am, am+1,;.---: 


zm 5 anz” = os Ont+mz”. (4) 


n>m n>0 


We combined operations A and B to solve the Fibonacci problem in the 
previous section: G(z) was the generating function for (Fn), zG(z) for (Fna-1), 
z2?G(z) for (Fa—2), and (1 — z — z*)G(z) for (Fa — Fa—1 — Fn—2). Then, since 
F, — Fa—ı — Fa—2 is zero when n > 2, we found that (1 — z — 2?°)G(2) is a 
polynomial. Similarly, given any linearly recurrent sequence, that is, a sequence 
where an = C1an—1 +- +Cman-m, the generating function will be a polynomial 


divided by (1 — c1z — +++ — Gmz"™). 

Let us consider the simplest example of all: If G(z) is the generating func- 
tion for the constant sequence 1,1,1,..., then zG(z) generates 0,1,1,..., so 
(1 — z)G(z) = 1. This gives us the simple but very important formula 

1 
eerie oe oe i (5) 


C. Multiplication. If G(z) is the generating function for ag,a1,... and H(z) 
is the generating function for bo, b),..., then 


G(z)H(z) = (ao + aiz + a22? +- - )(bo + bız + boz? +- z) 
= (aobo) + (aobı + a1bo)z + (aob2 + ab + agbo)z? +-->; 


thus G(z) H(z) is the generating function for the sequence co, c1, ..., where 


Cn = 5 Qkbn-k- (6) 
k=0 
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Equation (3) is a very special case of this. Another important special case occurs 
when each b, is equal to unity: 


l G(z) = ao + (ao + a1)z + (ao + a1 + a2)? +-+. (7) 


Here we have the generating function for the sums of the original sequence. 
The rule for a product of three functions follows from (6); F(2)G(z2) H(z) 
generates do, d1, d2, ..., where 


dn = 5 aibjCk. (8) 
i,j,k >0 
i+j+k=n 


The general rule for products of any number of functions (whenever this is 


meaningful) is 
II X ana" = 5 Ze 5 QOkolikı +++ (9) 


j20 k>0 n>0  ko,kı,...>0 
kotki +- =n 
When the recurrence relation for some sequence involves binomial coeffi- 
cients, we often want to get a generating function for a sequence co, c1, ... defined 
by 
n 
= J (p) oe (10) 
k 


In this case it is usually better to use generating functions for the sequences 
(an/n!), (bn/n!}, (Cn/n!), since we have 


ao, a e O ae bo , bi b2 i> C05 CL ee ee 
i ee on oi an =o a ot? i 
(11) 


where cn is given by Eq. (10). 


D. Change of z. Clearly G(cz) is the generating function for the sequence 
ao, ca1, C7a2,.... As a particular case, the generating function for 1,c,c?,c°,... 
is 1/(1 — cz). 


There is a familiar trick for extracting alternate terms of a series: 
53(G(z) = G(-z)) =a + aaz? + aaz po a 
3(G(2) g G(—z)) = az + azz? + a5zř + 


(12) 


Using complex roots of unity, we can extend this idea and extract every mth 
term: Let w = €27*/™ = cos(2r /m) + isin(27/m); we have 


1 — 
J ap” = a > w*Gwkz), O<r<m. (13) 
n>0,n mod m=r O0<k<m 
V3; 
2 


(See exercise 14.) For example, if m = 3 and r = 1, we have w = —4 + 1 


a complex cube root of unity; it follows that 


? 


az + agz* + azz" +++ = 3(G(z) + uG (wz) +w?Gw?2z)). 
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E. Differentiation and integration. The techniques of calculus give us further 
operations. If G(z) is given by Eq. (1), the derivative is 


G'(z) = ay + 2agz + 3agz7 +--+ = Sok + Lagsi2”. (14) 
k>0 


The generating function for the sequence (nan) is zG’(z). Hence we can combine 
the nth term of a sequence with polynomials in n by manipulating the generating 
function. 

Reversing the process, integration gives another useful operation: 


í 1 1 1 
f G(t) dt = aoz + ga zane H = J go-z". (15) 
k>1 
As special cases, we have the derivative and integral of (5): 
1 2 k 
k>0 
1 1 1 1 
r tae PS n (17) 
k>1 


We can combine the second formula with Eq. (7) to get the generating function 
for the harmonic numbers: 


1 1 E EEES ; 
ln =z+3 t5? pa a (18) 


F. Known generating functions. Whenever it is possible to determine the 
power series expansion of a function, we have implicitly found the generating 
function for a particular sequence. These special functions can be quite useful 
in conjunction with the operations described above. The most important power 
series expansions are given in the following list. 


i) Binomial theorem. 


(1+2) =1+rz4 trv su. ee (19) 


k>0 


When r is a negative integer, we get a special case already reflected in Eqs. (5) 
and (16): 
1 —n-1 k n+k\ k 
ae D k Jira) =>( n JA (an) 
k>0 k>0 
There is also a generalization, which was proved in exercise 1.2.6—25: 
—2t-1 

r(r ) z2 


e 


k>0 


ge =1+rz4 


+1 


if x is the continuous function of z that solves the equation ztt! = x! + z, where 


x = 1 when z=0. 
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ii) Exponential series. 


1 
expz=e7 =1l+z+i2?4---= 3". (22) 
In general, we have the following formula involving Stirling numbers: 
1 n+1 ky) 2 
2 4)yrn = n a n+1 a | { a 
(e ) i eat n he j eo nJ k! (23) 
iii) Logarithm series (see (17) and (18)). 


1 1 
Int he) See oh ge Ae a a (24) 


Gon m( >) =J Gs A (25) 


k>1 


Stirling numbers, as in (23), give us a more general equation: 


1 wo m 1 n+1 n+l _ oot He 
(mi) E so n | i = n] k! (=e 


Further generalizations, including many sums of harmonic numbers, appear in 
papers by D. A. Zave, Inf. Proc. Letters 5 (1976), 75-77; J. Spieß, Math. Comp. 
55 (1990), 839-863. 


iv) Miscellaneous. 


Het)... +n) =D, A, ta 
k 
2” k 
0-302)... (=n) Dnj (28) 
2 1 1 2 = Byz! 
ea 5% + 75% ee ~ (29) 


The coefficients B, that appear in the last formula are the Bernoulli numbers; 
they will be examined further in Section 1.2.11.2. A table of Bernoulli numbers 
appears in Appendix A. 

The next identity, analogous to (21), will be proved in exercise 2.3.4.4-29: 


2t kt)*-1 
walter E A ajna O a T 


2 
k>0 


if x is the continuous function of z that solves the equation xz = et, where x = 1 
when z = 0. Significant generalizations of (21) and (30) are discussed in exercise 
4.7-22. 
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G. Extracting a coefficient. It is often convenient to use the notation 


[2"] G(z) (31) 
for the coefficient of z” in G(z). For example, if G(z) is the generating function 
in (1) we have [z”] G(z) = an and [z”] G(z)/(1 — z) = yp ae. One of the 
most fundamental results in the theory of complex variables is a formula of 
A. L. Cauchy [Exercices de Math. 1 (1826), 95-113 = Œuvres (2) 6, 124-145, 
Eq. (11)], by which we can extract any desired coefficient with the help of a 


contour integral: 
1 G(z) dz 
GCO $6 SSS, (32) 


2ri 
if G(z) converges for z = z and 0 < r < |zo|. The basic idea is that $izl=r z™ dz 
is zero for all integers m except m = —1, when the integral is 


/ (re) d(re”) = i dO = 2ri. 


T T 


Equation (32) is of importance primarily when we want to study the approximate 
value of a coefficient. 


We conclude this section by returning to a problem that was only partially 
solved in Section 1.2.3. We saw in Eq. 1.2.3-(13) and exercise 1.2.3-29 that 


1 n 2 1 n 
2 |. 
2 Titj= 3 (£a) T3 a2 
1<i<j<n k=1 k=1 
1 n 3 1 n n 1 n 
L wasee=§ (oe) +5 (Dee) (Doak) +22) 
1<i<j<k<n k=1 k=1 k=1 k=1 
In general, suppose that we have n numbers 21, £2,..., gn and we want the 
sum 
hm = 5 Tj ee Vj, (33) 
1<j1 L Ljm <n 
If possible, this sum should be expressed in terms of S1, S2, ..., Sm, where 


S=¢_ 2 (34) 


k=1 


the sum of jth powers. Using this more compact notation, the formulas above 
become h = $57 + 452; hg = ES} + 49192 + $93. 
We can attack this problem by setting up the generating function 


G(z) =1t mz + hoz? +- = So hez". (35) 
k>0 
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By our rules for multiplying series, we find that 
G(z) = (1 + z112 +22? +- (1 + r22 + r22 +--+)... (1+ Enz +z? +) 
1 
= . 6 
(1 — z1z)(1 — x22)... (L — az) (36) 


So G(z) is the reciprocal of a polynomial. It often helps to take the logarithm 
of a product, and we find from (17) that 


l =] ee | 
Ao a aa i "Tanz 
kk k „k k (37) 
_ y ziz OR 3 Lz _ Skz 
k>1 k>1 k>1 


Now In G(z) has been expressed in terms of the $’s; so all we must do to obtain 
the answer to our problem is to compute the power series expansion of G(z) 
again, with the help of (22) and (9): 


k 
G(z) = e™F®) = exp 2 ae ) _ Il eSrz"/k 


k21 k>1 
sit See? Shah 
= (145124 y +e’ 14 7 Fazat von 
se pe Skm = 
7 ( Di 1kiky! Qkekol °° mkmk,! as (38) 
m20 k1,k2,....km>0 1: 2! m! 


ki+2kə+ + Mkm=m 


The parenthesized quantity is hm. This rather imposing sum is really not 
complicated when it is examined carefully. The number of terms for a particular 
value of m is p(m), the number of partitions of m (Section 1.2.1). For example, 
one partition of 12 is 
12=54+2424+2+41; 

this corresponds to a solution of the equation kı + 2k2 +---+12k,2 = 12, where 
kj is the number of j7’s in the partition. In our example kı = 1, k2 = 3, k5 = 1, 
and the other k’s are zero; so we get the term 

Sı S3 Ss 1 

= Si S35) 

111! 233! 5111 24017275 
as part of the expression for hig. By differentiating (37) it is not difficult to 
derive the recurrence 


1 
hy = z (Shni + S2hn-2 +°: + Snho), nzi. (39) 


An enjoyable introduction to the applications of generating functions has 
been given by G. Pólya, “On picture writing,” AMM 63 (1956), 689-697; his 
approach is continued in CMath, Chapter 7. See also the book generatingfunc- 
tionology by H. S. Wilf, second edition (Academic Press, 1994). 
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A generating function is a clothesline 
on which we hang up a sequence of numbers for display. 


— H. S. WILF (1989) 


EXERCISES 
1. [M12] What is the generating function for the sequence 2, 5, 13,35, ... = (2"4+3")? 
> 2. [M13] Prove Eq. (11). 


3. [HM21] Differentiate the generating function (18) for (Hn), and compare this with 
the generating function for (ono Hy). What relation can you deduce? 


4. [M01] Explain why Eq. (19) is a special case of Eq. (21). 
5. [M20] Prove Eq. (23) by induction on n. 
> 6. [HM15] Find the generating function for 


differentiate it and express the coefficients in terms of harmonic numbers. 
7. [M15] Verify all the steps leading to Eq. (38). 
8. [M23] Find the generating function for p(n), the number of partitions of n. 


9. [M11] In the notation of Eqs. (34) and (35), what is h4 in terms of S1, S2, S3, 
and S4? 


> 10. [M25] An elementary symmetric function is defined by the formula 
em = Lip ess Lim 
1<ji<i-<jm<n 


(This is the same as hm of Eq. (33), except that equal subscripts are not allowed.) Find 
the generating function for em, and express em in terms of the S; in Eq. (34). Write 
out the formulas for e1, e2, e3, and e4. 


> 11. [M25] Equation (39) can also be used to express the S’s in terms of the h’s: We 
find Sı = hi, S2 = 2h2 — hi, S3 = 3h3 — 3hıh2 + hï, etc. What is the coefficient of 
nk ph, od hEr in this representation of Sm, when kı + 2k2 +---+mkm =m? 


> 12. [M20] Suppose we have a doubly subscripted sequence (amn) for m,n = 0, 1,...; 
show how this double sequence can be represented by a single generating function of 


two variables, and determine the generating function for ((2)) 


13. [HM22] The Laplace transform of a function f(x) is the function 


Lf(s) = f e“ f(t) dt. 


Given that ao, a1,a@2,... is an infinite sequence having a convergent generating function, 
let f(x) be the step function 5°, ax [0 < k < x]. Express the Laplace transform of f(x) 
in terms of the generating function G for this sequence. 


14. [HM21] Prove Eq. (13). 
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15. [M28] By considering H(w) = >7,,59 Gn(z)w", find a closed form for the gener- 
ating function E 


Ga (2) -5("7 A =o Poca" 


16. [M22] Give a simple formula for the generating function Gnr(z) = X, ankr”, 
where ankr is the number of ways to choose k out of n objects, subject to the condition 
that each object may be chosen at most r times. (If r = 1, we have (7) ways, and if 
r > k, we have the number of combinations with repetitions as in exercise 1.2.6-60.) 


17. [M25] What are the coefficients of 1/(1 — z)” if this function is expanded into a 
double power series in terms of both z and w? 

18. [M25] Given positive integers n and r, find a simple formula for the value of the 
following sums: (a) ee ee kiko... kr; (b) ee ee kiko... Kp. 
(For example, when n = 3 and r = 2 the sums are, respectively, 1-2+1-3+4+2-3 and 
1-14+1-241-34+2-2+4+2-3+43-3.) 

19. [HM32] (C. F. Gauss, 1812.) The sums of the following infinite series are well 
known: 


I t I I £ of T 
1 } H- =]ln2; 1 } peric =; 
4 . 3°5 7 4 
ee eS a i, 
4 10 9 
Using the definition 
n-p- 
n n+r 
n>1 


found in the answer to exercise 1.2.7—24, these series may be written respectively as 
1 2 1 1 3 1 1 
1— =Ay 79; —— —H — 3/43 —_— -H = A/3. 
g 1/2; 371 1/4 + 4773/43 1.6 1/6 + g 2/3 


Prove that, in general, H,/q has the value 


q 


2pk k 
T cot Ër In2q +2 > cos “Pr r. nsin €r, 
2 q O<k< q q 
q/2 


when p and q are integers with 0 < p < q. [Hint: By Abel’s limit theorem the sum is 
1 1 
lim ` (+- pa 
z>1- £ \n n+ p/q 


Use Eq. (13) to express this power series in such a way that the limit can be evaluated. ] 
20. [M21] For what coefficients cmx is X >o n™2” = Pko Cme2z*/(1 — z)*t*2 


21. [HM30] Set up the generating function for the sequence (n!) and study properties 
of this function. 


22. [M21] Find a generating function G(z) for which 


BIC aae ea a Ce) i) Ce) 


ko+2k1+4ko+8k3+--=n 
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23. [M33] (L. Carlitz.) (a) Prove that for all integers m > 1 there are polynomials 


fm(21,.--,2m) and gm(21,.--,;2m) such that the formula 

> ( r )( kı ) hues Veg km 

hi kaso SO ki Re i km) 
= fm irm) galis m) 


is an identity for all integers n > r > 0. 
(b) Generalizing exercise 15, find a closed form for the sum 


= kı ko km ky km 
Sn(21,---52m) = 5 an aa ee bs eget 


k1,...,km>0 
in terms of the functions fm and gm in part (a). 
(c) Find a simple expression for S,(z1,...,2m) when 21 =-:: = Zm = 2. 
24. [M22] Prove that, if G(z) is any generating function, we have 
m n— n m 
(7) ICO = 210+ e6)”. 
k 
Evaluate both sides of this identity when G(z) is (a) 1/(1 — z); (b) (e7 — 1)/z. 
25. [M23] Evaluate the sum X>, (7) Co (—2)* by simplifying the equivalent for- 
mula >>, [w*] (1 — 2w)” [z7] (1 + 2)?77?*. 
26. [M40] Explore a generalization of the notation (31) according to which we might 
write, for example, [22 — 2z°] G(z) = a2 — 2a5 when G(z) is given by (1). 


1.2.10. Analysis of an Algorithm 

Let us now apply some of the techniques of the preceding sections to the study 
of a typical algorithm. 

Algorithm M (Find the maximum). Given n elements X [1], X[2],...,X[n], we 
will find m and j such that m = X[j] = maxi<i<n X[i], where j is the largest 
index that satisfies this relation. 


M1. [Initialize] Set j 4 n, k + n — 1, m 4+ X[n]. (During this algorithm we 
will have m = X [j] = maxk<i<n X [i].) 

M2. [All tested?] If k = 0, the algorithm terminates. 

M3. [Compare.] If X[k] < m, go to M5. 


M4. [Change m.] Set j + k, m + X[k]. (This value of m is a new current 
maximum.) 


M5. [Decrease k.] Decrease k by one and return to M2. I 


This rather obvious algorithm may seem so trivial that we shouldn’t bother 
to analyze it in detail; but it actually makes a good demonstration of the way 
in which more complicated algorithms may be studied. Analysis of algorithms 
is quite important in computer programming, because there are usually several 
algorithms available for a particular application and we would like to know which 
is best. 
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n—-1 


y No 
F M1. Initialize Fue. All tested? (M3. Compare) -> M4. Change m re M5. Decrease k 
n—1 ZN 


1| Yes 


m> X[k] n—1—A 


y 


Fig. 9. Algorithm M. Labels on the arrows indicate the number of times each path 
is taken. Note that “Kirchhoff’s first law” must be satisfied: The amount of flow into 
each node must equal the amount of flow going out. 


Algorithm M requires a fixed amount of storage, so we will analyze only the 
time required to perform it. To do this, we will count the number of times each 
step is executed (see Fig. 9): 


Step number Number of times 


M1 1 
M2 n 
M3 n— 1 
M4 A 
M5 n— 1 


Knowing the number of times each step is executed gives us the information 
necessary to determine the running time on a particular computer. 

In the table above we know everything except the quantity A, which is the 
number of times we must change the value of the current maximum. To complete 
the analysis, we shall study this interesting quantity A. 

The analysis usually consists of finding the minimum value of A (for op- 
timistic people), the maximum value of A (for pessimistic people), the average 
value of A (for probabilistic people), and the standard deviation of A (a quanti- 
tative indication of how close to the average we may expect the value to be). 

The minimum value of A is zero; this happens if 


X[n] = max X[k]. 


The mazimum value is n — 1; this happens in case 
X[1] > X[2] >- > X[n]. 


Thus the average value lies between 0 and n — 1. Is it $n? Is it yn? 
To answer this question we need to define what we mean by the average; and 
to define the average properly, we must make some assumptions about the 
characteristics of the input data X[1], X[2],...,X[n]. We will assume that the 
X[k] are distinct values, and that each of the n! permutations of these values 
is equally likely. (This is a reasonable assumption to make in most situations, 
but the analysis can be carried out under other assumptions, as shown in the 
exercises at the end of this section.) 
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The performance of Algorithm M does not depend on the precise values 
of the X[k]; only the relative order is involved. For example, if n = 3 we are 
assuming that each of the following six possibilities is equally probable: 


Situation Value of A Situation Value of A 
X[1] < X[2] < X[3] 0 X[2] < X[3] < X[1] 1 
X[1] < X[3] < X[2] 1 X[3] < X[1] < X12] 1 
X[2] < X[1] < X[3] 0 X[3] < X[2] < X[1] 2 


The average value of A when n = 3 comes to (0+1 +0+ 1+1 +2)/6 = 5/6. 

It is clear that we may take X[1], X[2], ..., X[n] to be the numbers 1,2,...,n 
in some order; under our assumption we regard each of the n! permutations as 
equally likely. The probability that A has the value k will be 


Pnk = (number of permutations of n objects for which A = k)/n!. (a) 


For example, from our table above, p30 Z, p31 4, p32 z. 


The average (“mean” or “expected”) value is defined, as usual, to be 


k 


The variance Vn is defined to be the average value of (A— An )?; we have therefore 
Vn = Silk =< An) Pnk -= 5 k’ Pnk a 2 Ay 5 kPnk + 42 X pnk 
k k k k 


=X pak — 2AnAn + A2 = Sk pnk — A2. (3) 
k k 


Finally, the standard deviation on is defined to be al Vins 

The significance of on can perhaps best be understood by noting that, for 
all r > 1, the probability that A fails to lie within ron of its average value is 
less than 1/r?. For example, |A — An| > 20, with probability < 1/4. (Proof: 
Let p be the stated probability. Then if p > 0, the average value of (A — An)? 
is more than p- (ron)? + (1 — p) - 0; that is, Vp > pr?°Vn.) This is usually called 
Chebyshev’s inequality, although it was actually discovered first by J. Bienaymé 
[Comptes Rendus Acad. Sci. 37 (Paris, 1853), 320-321]. 

We can determine the behavior of A by determining the probabilities pyx. 
It is not hard to do this inductively: By Eq. (1) we want to count the number of 
permutations on n elements that have A = k. Let this number be Pak = n! Pnk- 

Consider the permutations £1£2... £n on {1,2,...,n}, as in Section 1.2.5. 
If zı = n, the value of A is one higher than the value obtained on x2...%n; if 
xı Æ n, the value of A is exactly the same as its value on x2...%,. Therefore 
we find that Pak = Pin-1)(k-1) + (n — 1)Pin-1)k, OF equivalently 


_ il T n-—1 (4) 
Pnk = n Pim-V(k-1) a P(n-1)k: 4 
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Fig. 10. Probability distribution for step M4, when n = 12. The mean is 58301/27720, 
or approximately 2.10. The variance is approximately 1.54. 


This equation will determine p,, if we provide the initial conditions 


Pik = Sok; Pnk =O if kk <0. (5) 


We can now get information about the quantities pnk by using generating 
functions. Let 


k 


We know that A < n— 1, so Png = 0 for large values of k; thus G,,(z) is actually 

a polynomial, even though an infinite sum has been specified for convenience. 
From Eq. (5) we have G4 (z) = 1; and from Eq. (4) we have 

n— 1 z+n-1 

Gn-1(2) n = Gn-1(2). (7) 


n 


Gn(2) =  Ga-1(2)+ 


(The reader should study the relation between Eqs. (4) and (7) carefully.) We 
can now see that 


z+n-1 z+n-—-1l z+n-2 
Gn (2z) = ~ Gn-1(2) — a ni Gn_-2(z) =... 
1 
= tn-1)(z+n-2)...(z+1) 
1 ztn 
=a n ). (8) 


So G,,(z) is essentially a binomial coefficient! 
This function appears in the previous section, Eq. 1.2.9-(27), where we have 


colo BL 


Therefore png can be expressed in terms of Stirling numbers: 


sekaa o 


Figure 10 shows the approximate sizes of pnk when n = 12. 
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Now all we must do is plug this value of p,, into Eqs. (2) and (3) and we 
have the desired average value. But this is easier said than done. It is, in fact, 
unusual to be able to determine the probabilities Pn explicitly; in most problems 
we will know the generating function G,,(z), but we will not have any special 
knowledge about the actual probabilities. The important fact is that we can 
determine the mean and variance easily from the generating function itself. 

To see this, let’s suppose that we have a generating function whose coeffi- 
cients represent probabilities: 


G(z) =pot+pizt+ p22? +--+- 


Here pp is the probability that some event has a value k. We wish to calculate 
the quantities 


mean(G) = 5 kpk, var(G) = 5 k?’ pr — (mean(G))”. (10) 
k k 


Using differentiation, it is not hard to discover how to do this. Note that 
G(1)=1, (11) 


since G(1) = po + pı + p2 +--+ is the sum of all possible probabilities. Similarly, 
since G’(z) = >, kppz*~1, we have 


mean(G) = X` kpr = G'(1). (12) 
k 


Finally, we apply differentiation again and we obtain (see exercise 2) 


var(G) = G") + G’(1) — G'(1)’. (13) 


Equations (12) and (13) give the desired expressions of the mean and variance 
in terms of the generating function. 
In our case, we wish to calculate G! (1) = An. From Eq. (7) we have 


1 ztn-1 
ane) = —Gn_1(2) + 2 —et_(2); 


1 
G,(1) = — + Gn- (1). 
n 
From the initial condition G‘(1) = 0, we find therefore 


An = GI,(L) = Hp- 1. (14) 


This is the desired average number of times step M4 is executed; it is approxi- 
mately Inn when n is large. [Note: The rth moment of A+1, namely the quantity 
(K+ DY" pak, is [°] (1 — 2) 1d, {4} (ln 4)’, and it has the approximate 
value (Inn)"; see P. B. M. Roes CACM 9 (1966), 342. The distribution of A 
was first studied by F. G. Foster and A. Stuart, J. Roy. Stat. Soc. B16 (1954), 
1-22.] 

We can proceed similarly to calculate the variance Vp. Before doing this, 
let us state an important simplification: 
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Theorem A. Let G and H be two generating functions with G(1) = H(1) = 1. 
If the quantities mean(G) and var(G) are defined by Eqs. (12) and (13), we have 


mean(GH) = mean(G) + mean(#); var(GH) = var(G) + var(H). (a5) 


We will prove this theorem later. It tells us that the mean and variance of 
a product of generating functions may be reduced toa sum. J 


Letting Qn(z) = (z +n — 1)/n, we have Q/,(1) = 1/n, Q! (1) = 0; hence 


mean(Q,) = . var (Qn) = > = 2 


n n? 
Finally, since Gn (2) = [Ik Qk(2), it follows that 


Summing up, we have found the desired statistics related to quantity A: 
A= (min 0, ave Hn— 1, maxn-—1, dev Ha- HÊ) : (16) 


The notation used in Eq. (16) will be used to describe the statistical character- 
istics of other probabilistic quantities throughout this book. 

We have completed the analysis of Algorithm M; the new feature that has 
appeared in this analysis is the introduction of probability theory. Elementary 
probability theory is sufficient for most of the applications in this book: The 
simple counting techniques and the definitions of mean, variance, and standard 
deviation already given will answer most of the questions we want to ask. More 
complicated algorithms will help us develop an ability to reason fluently about 
probabilities. 

Let us consider some simple probability problems, to get a little more 
practice using these methods. In all probability the first question that comes 
to mind is a coin-tossing problem: Suppose we flip a coin n times and there is a 
probability p that heads turns up after any particular toss; what is the average 
number of heads that will occur? What is the standard deviation? 

We will consider our coin to be biased; that is, we will not assume that 
p= L, This makes the problem more interesting, and, furthermore, every real 
coin is biased (or we could not tell one side from the other). 

Proceeding as before, we let png be the probability that k heads will occur, 
and let Gn (z) be the corresponding generating function. We have clearly 


Pnk = PP(n-1)(k-1) + 4 P(n—1)k» (17) 


where q = 1 — pis the probability that tails turns up. As before, we argue from 
Eq. (17) that G,(z) = (q + pz)Gn_1(z); and from the obvious initial condition 
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Gi(z) = q + pz we have 
Gn(z) = (q + pz)”. (18) 
Hence, by Theorem A, 
mean(Gn) = n mean(G1) = pn; 
var(Gn) = n var(G1) = (p — p”)n = pan. 

For the number of heads, we have therefore 

(min 0, ave pn, maxn, dev ypqn). (19) 
Figure 11 shows the values of png when p = 3, n = 12. When the standard devi- 
ation is proportional to „y/n and the difference between maximum and minimum 
is proportional to n, we may consider the situation “stable” about the average. 


0 1 2 3 4 5 6 7 8 9 10 11 12 


Fig. 11. Probability distribution for coin-tossing: 12 independent tosses with a chance 
of success equal to 3/5 at each toss. 


Let us work one more simple problem. Suppose that in some process there 
is equal probability of obtaining the values 1,2,...,n. The generating function 
for this situation is 

1, 1 grtl_ x 


1 1 
Glz) = —2+ —2? +--++4+ —2” 
n n n n z-1 


(20) 


We find after some rather laborious calculation that 
nz — (n+1)2"+1 
n(z — 1)? 
n(n —1)2z"41 — 2(n + 1)(n — 1)z” + n(n + 1)z""1—-2 
n(z — 1)’ f 


G (z) = 


G” (z) = 


Now to calculate the mean and variance, we need to know G’(1) and G” (1); but 
the form in which we have expressed these equations reduces to 0/0 when we 
substitute z = 1. This makes it necessary to find the limit as z approaches unity, 
and that is a nontrivial task. 

Fortunately there is a much simpler way to proceed. By Taylor’s theorem 


we have wt 
OP bos (21) 


G(1+2) =G(1)+G(1)z+ 
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therefore we merely have to replace z by z+1 in (20) and read off the coefficients: 


1(1l+2)"*1-1-z mel p MADn a 
2 6 


GA +2)= 7 =1+ 


Fess 
z 


It follows that G’(1) = $(n +1), G’(1) = (n + 1)(n — 1), and the statistics for 
the uniform distribution are 


n+1 (n+1)(n—1) 
z> «max Nn, dev ee 


(min 1, ave (22) 
In this case the deviation of approximately 0.289n gives us a recognizably un- 
stable situation. 


We conclude this section by proving Theorem A and relating our notions 
to classical probability theory. Suppose X is a random variable that takes on 
only nonnegative integer values, where X = k with probability p. Then G(z) = 
Po + piz+ p22? +--: is called the probability generating function for X, and the 
quantity G(e"*) = po+pie”+p2e?+--- is conventionally called the characteristic 
function of this distribution. The distribution given by the product of two such 
generating functions is called the convolution of the two distributions, and it 
represents the sum of two independent random variables belonging to those 
respective distributions. 

The mean or average value of a random quantity X is often called its expected 
value, and denoted by EX. The variance of X is then EX?—(EX)?. Using this 
notation, the probability generating function for X is G(z) = Ez*, the expected 
value of z*, in cases when X takes only nonnegative integer values. Similarly, 
if X is a statement that is either true or false, the probability that X is true is 
Pr(X) = E[X], using Iverson’s convention (Eq. 1.2.3-(16)). 

The mean and variance are just two of the so-called semi-invariants or cu- 
mulants introduced by T. N. Thiele in 1889 [see A. Hald, International Statistical 
Review 68 (2000), 137-153]. The semi-invariants «K1, K2,3,... are defined by 
the rule 


1! 2! gp t= in Ge), (23) 
We have ai 
n= — l t ; 
a= ee | 
in particular, 
etG' (et) 
= =G'(1 
Be, lesa (1) 


because G(1) = 0, pk = 1, and 

eG" (et) T etG" (et) e”%G" (et)? 
Ge’) Ge’) Ge’)? 

Since the semi-invariants are defined in terms of the logarithm of a generating 


function, Theorem A is obvious, and, in fact, it can be generalized to apply to 
all of the semi-invariants. 


9 2 FMF") - ey. 


K2 = 
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A normal distribution is one for which all semi-invariants are zero except 
the mean and variance. In a normal distribution, we can improve significantly 
on Chebyshev’s inequality: The probability that a normally distributed random 
value differs from the mean by less than the standard deviation is 


1 
R et /2 dt, 
V2T J-1 


that is, about 68.268949213709% of the time. The difference is less than twice 
the standard deviation about 95.449973610364% of the time, and it is less than 
three times the standard deviation about 99.730020393674% of the time. The 
distributions specified by Eqs. (8) and (18) are approximately normal when n is 
large (see exercises 13 and 14). 

We often need to know that a random variable is unlikely to be much larger 
or smaller than its mean value. Two extremely simple yet powerful formulas, 
called the tail inequalities, provide convenient estimates of such probabilities. 
If X has the probability generating function G(z), then 


Pr(X <r) < x "G(az) forO<a<1,; (24) 
Pr(X >r) < a "G(az) for x > 1. (25) 
The proofs are easy: If G(z) = po + piz + p22? +--+, we have 
Pr(X < r) = po+pi ++ pir] <a "pote "pit: +a'! "pp <a "G(a) 
when 0 < x <1, and 
Pr(X > r) =p + Pig t Ss sl pr + gpa +--+ <a G(x) 


when x > 1. By choosing values of « that minimize or approximately minimize 
the right-hand sides of (24) and (25), we often obtain upper bounds that are 
fairly close to the true tail probabilities on the left-hand sides. 

Exercises 21-23 illustrate the tail inequalities in several important cases. 
These inequalities are special cases of a general principle pointed out by A. N. 
Kolmogorov in his book Grundbegriffe der Wahrscheinlichkeitsrechnung (1933): 
If f(t) > s > 0 for all t > r, and if f(t) > 0 for all t in the domain of the random 
variable X, then Pr(X > r) < s71 E f(X) whenever E f(X) exists. We obtain 
(25) when f(t) = zt and s = x”. [S. Bernstein had contributed key ideas in 
Uchenye zapiski Nauchno-Issledovatel’skikh kafedr Ukrainy 1 (1924), 38-48.] 


EXERCISES 


1. [10] Determine the value of pno from Eqs. (4) and (5) and interpret this result 
from the standpoint of Algorithm M. 

2. [HM16] Derive Eq. (13) from Eq. (10). 

3. [M15] What are the minimum, maximum, average, and standard deviation of 
the number of times step M4 is executed, if we are using Algorithm M to find the 
maximum of 1000 randomly ordered, distinct items? (Give your answer as decimal 
approximations to these quantities.) 

4. [M10] Give an explicit, closed formula for the values of png in the coin-tossing 
experiment, Eq. (17). 
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5. [M13] What are the mean and standard deviation of the distribution in Fig. 11? 


6. [HM27| We’ve computed the mean and the variance of the important probability 
distributions (8), (18), (20). What is the third semi-invariant, «3, in each of those cases? 


7. [M27] In our analysis of Algorithm M, we assumed that all the X[k] were distinct. 
Suppose, instead, that we make only the weaker assumption that X[1], X[2],...,X[n] 
contain precisely m distinct values; the values are otherwise random, subject to this 
constraint. What is the probability distribution of A in this case? 


8. [M20] Suppose that each X[k] is taken at random from a set of M distinct 
elements, so that each of the M” possible choices for X [1], X[2],...,X[n] is considered 
equally likely. What is the probability that all the X[k] will be distinct? 


9. [M25] Generalize the result of the preceding exercise to find a formula for the 
probability that exactly m distinct values occur among the X’s. Express your answer 
in terms of Stirling numbers. 


10. [M20] Combine the results of the preceding three exercises to obtain a formula 
for the probability that A = k under the assumption that each X is selected at random 
from a set of M objects. 


11. [M15] What happens to the semi-invariants of a distribution if we change G(z) 
to F(z) = z"G(z)? 

12. [HM21] When G(z) = po + pız + pez? +--+ represents a probability distribution, 
the quantities M, = >>, k"p, and mn = }_,(k — M1)" px are called the “nth moment” 
and “nth central moment,” respectively. Show that G(e’) = 1+ Mit + Mot?/2! +---; 
then use Arbogast’s formula (exercise 1.2.5-21) to show that 


—1)Ri tkt thn Tal (ky + kg +: + kp — 1 
pe £ (—1) n! (ky + ko + + ) 


ve më? ME” 
kı! 1!%1 kg! 212 .. ky! n!Fn ng Nees 


k1,ko,...,kn 20 
kit2ko+--- =n 


In particular, xı = Mı, k2 = Mo—M? (as we already knew), k3 = M3—3M,M2+2M?, 
and k4 = Ma4 — 4Mı M3 + 12M? M2 — 3M2 — 6Mf. What are the analogous expressions 
for Kn in terms of the central moments m2, m3, ..., when n > 2? 


13. [HM38] A sequence of probability generating functions Gn(z) with means un and 
deviations on is said to approach a normal distribution if 

lim eenn Gg (ett/on) = et /2 

noo 2 
for all real values of t. Using Gn(z) as given by Eq. (8), show that Gn(z) approaches 
a normal distribution. 

Note: “Approaching the normal distribution,” as defined here, can be shown to 

be equivalent to the fact that 


lim Pr (= Hn < 2) oe f gr dt, 

n— oo On VIT ees 
where Xn is a random quantity whose probabilities are specified by Gn(z). This is 
a special case of P. Lévy’s important “continuity theorem,” a basic result in mathe- 
matical probability theory. A proof of Lévy’s theorem would take us rather far afield, 
although it is not extremely difficult [for example, see Limit Distributions for Sums of 
Independent Random Variables by B. V. Gnedenko and A. N. Kolmogorov, translated 
by K. L. Chung (Reading, Mass.: Addison-Wesley, 1954)]. 
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14. [HM30] (A. de Moivre.) Using the conventions of the previous exercise, show that 
the binomial distribution G,(z) given by Eq. (18) approaches the normal distribution. 
15. [HM23] When the probability that some quantity has the value k is e~"(u*/k!), 
it is said to have the Poisson distribution with mean p. 

a) What is the generating function for this set of probabilities? 

b) What are the values of the semi-invariants? 

c) Show that as n — oo the Poisson distribution with mean np approaches the normal 

distribution in the sense of exercise 13. 

16. [M25] Suppose X is a random variable whose values are a mixture of the proba- 
bility distributions generated by gi(z), g2(z), ---, gr(z), in the sense that it uses gi,(z) 
with probability pk, where pı + p2 +--+ pr = 1. What is the generating function 
for X? Express the mean and variance of X in terms of the means and variances of gi, 
G2, -++5 Qr. 
17. [M27] Let f(z) and g(z) be generating functions that represent probability dis- 
tributions. 


a) Show that h(z) = g(f(z)) is also a generating function representing a probability 
distribution. 
b) Interpret the significance of h(z) in terms of f(z) and g(z). (What is the meaning 
of the probabilities represented by the coefficients of h(z)?) 
c) Give formulas for the mean and variance of h in terms of those for f and g. 
18. [M28] Suppose that the values taken on by X[1], X[2], ..., X[n] in Algorithm M 
include exactly ki ones, k2 twos, ..., kn n’s, arranged in random order. (Here 


ki +k2 +- + kn =n. 


The assumption in the text is that kı ko tee kn 1.) Show that in this 
generalized situation, the generating function (8) becomes 


(+5) kn—12 + =) (1 + kn—i + =) kiz + ka +- + =) 

Rn ( kni Fkn ) \Rn—2+ hai thn J Gee [> 
using the convention 0/0 = 1. 

19. [M21] If a, > a; for 1 < j < k, we say that a, is a left-to-right mazimum of the 
sequence a1 G2... Gn. Suppose ai a2... An is a permutation of {1,2,...,n}, and let 
bı b2 ... bn be the inverse permutation, so that a, = l if and only if b; = k. Show that 
ax is a left-to-right maximum of ai a2 ... an if and only if k is a right-to-left minimum 
of bı be shana bn. 

20. [M22] Suppose we want to calculate max{|a1 — bi|, |a2 — b2|, . -. , |an — bn|} when 
by < b2 < --- < bn. Show that it is sufficient to calculate max{mz, mr}, where 


mr, = max{a, — by | ay is a left-to-right maximum of aiaz... an} : 
MR = max{bp — ax | ax is a right-to-left minimum of aja2... an} : 


(Thus, if the a’s are in random order, the number of k’s for which a subtraction must 
be performed is only about 21n n.) 


21. [HM21] Let X be the number of heads that occur when a random coin is flipped 
n times, with generating function (18). Use (25) to prove that 


Pr(X > n(p + €)) < ene 7 (29) 


when € > 0, and obtain a similar estimate for Pr(X < n(p — €)). 
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> 22. [HM22] Suppose X has the generating function (qı + p1z)(q2+p2z).-- (dn +Pn2), 
where pp + qr = 1 for 1 < k < n. Let u = E X =pi+p2+---+ pn. (a) Prove that 


Pr(X < pr) < (r "e"7!)", when 0 <r <1; 

Pr(X > ur) < (r7"e"')", when r > 1. 
(b) Express the right-hand sides of these estimates in convenient form when r ~ 1. 
(c) Show that if r is sufficiently large we have Pr(X > pr) < 27*". 


23. [HM23] Estimate the tail probabilities for a random variable that has the negative 
binomial distribution generated by (q — pz) ”, where q = p+ 1. 


*1.2.11. Asymptotic Representations 


We often want to know a quantity approximately, instead of exactly, in order to 
compare it to another. For example, Stirling’s approximation to n! is a useful 
representation of this type, when n is large, and we have also made use of the 
fact that Hn ~ lnn + y. The derivations of such asymptotic formulas generally 
involve higher mathematics, although in the following subsections we will use 
nothing more than elementary calculus to get the results we need. 


*1.2.11.1. The O-notation. Paul Bachmann introduced a very convenient 
notation for approximations in his book Analytische Zahlentheorie (1894). It is 
the O-notation, which allows us to replace the “~” sign by “=” and to quantify 
the degree of accuracy; for example, 


1 
Hy =Inn+7+0(—). (1) 


(Read, “H sub n equals the natural log of n plus Euler’s constant [pronounced 
‘Oiler’s constant’] plus big-oh of one over n.”) 

In general, the notation O(f(n)) may be used whenever f(n) is a function 
of the positive integer n; it stands for a quantity that is not explicitly known, 
except that its magnitude isn’t too large. Every appearance of O(f(n)) means 
precisely this: There are positive constants M and ng such that the number £n 
represented by O(f(n)) satisfies the condition |zn| < M|f(n)|, for all integers 
n > no. We do not say what the constants M and no are, and indeed those 
constants are usually different for each appearance of O. 

For example, Eq. (1) means that |H, — lnn — y| < M/n when n > no. 
Although the constants M and no are not stated, we can be sure that the quantity 
O(1/n) will be arbitrarily small if n is large enough. 

Let’s look at some more examples. We know that 


1742? +- n? = n(n + §)(n +1) = in? + in? + En; 


so it follows that 


124.22 n? = O(n’), (2) 
17497 4...4n7 = O(n*), (3) 
12 4 92 eran oe n2 = in + O(n?) (4) 
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Equation (2) is rather crude, but not incorrect; Eq. (3) is a stronger statement; 
and Eq. (4) is stronger yet. To justify these equations we shall prove that if 
P(n) = ao + an +--+- + amn” is any polynomial of degree m or less, then we 
have P(n) = O(n™). This follows because 


|P(n)| < lao] + laa] +--+ + Jam|n™ = (Jao| /n™ + Jay] /n™-* +++ + |am|)n™ 
< (lao + lar] +--+ + |am|)n™, 


when n > 1. So we may take M = |ao| + Jai| +--+: + |am| and no = 1. Or we 
could take, say, M = |ao|/2™ + |ai|/2”~1 +--+ + |am| and no = 2. 

The O-notation is a big help in approximation work, since it describes briefly 
a concept that occurs often and it suppresses detailed information that is usually 
irrelevant. Furthermore, it can be manipulated algebraically in familiar ways, 
although certain important differences need to be kept in mind. The most impor- 
tant consideration is the idea of one-way equalities: We write $n? +n = O(n’), 
but we never write O(n?) = $n? +n. (Or else, since $n? = O(n), we might 
come up with the absurd relation in? = in? +n.) We always use the convention 
that the right-hand side of an equation does not give more information than the 
left-hand side; the right-hand side is a “crudification” of the left. 

This convention about the use of “=” may be stated more precisely as 
follows: Formulas that involve the O(f(n))-notation may be regarded as sets 
of functions of n. The symbol O(f(n)) stands for the set of all functions g of 
integers such that there exist constants M and no with |g(n)| < M|f(n)| for 
all integers n > no. If S and T are sets of functions, then S + T denotes the 
set {g +h | g E€ Sandh E T}; we define S + c, S—T, S-T, log S, etc., ina 
similar way. If a(n) and (n) are formulas that involve the O-notation, then 
the notation a(n) = (n) means that the set of functions denoted by a(n) is 
contained in the set denoted by 8(n). 

Consequently we may perform most of the operations we are accustomed to 
doing with the “=” sign: If a(n) = (n) and B(n) = y(n), then a(n) = y(n). 
Also, if a(n) = (n) and if 6(n) is a formula resulting from the substitution of 
b(n) for some occurrence of a(n) in a formula y(n), then y(n) = 6(n). These two 
statements imply, for example, that if g(£1, £2, .. . , £m) is any real function what- 
ever, and if ag(n) = x(n) for 1 < k < m, then g(aj(n),a9(n),...,Qm(n)) = 
9(Bi(n), Ba(n),... ,Bm(n)). 


Here are some of the simple operations we can do with the O-notation: 


f(n) = O(fF(n)), (5) 

c- O(f(n)) = O(f(n)), if c is a constant, (6) 
O(f(n)) + O(f(n)) = O(F(n)), (7) 
O(O(F (n) = OF), (8) 
O(f(n)) O(g(n)) = O(F(n)g(n)), (9) 
O(f(n)g(n)) = f(n)O(g(n)) (10) 
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The O-notation is also frequently used with functions of a complex variable z, 
in the neighborhood of z = 0. We write O(f(z)) to stand for any quantity g(z) 
such that |g(z)| < M |f(z)| whenever |z| < r. (As before, M and r are unspecified 
constants, although we could specify them if we wanted to.) The context of O- 
notation should always identify the variable that is involved and the range of 
that variable. When the variable is called n, we implicitly assume that O( f (n)) 
refers to functions of a large integer n; when the variable is called z, we implicitly 
assume that O(f(z)) refers to functions of a small complex number z. 

Suppose that g(z) is a function given by an infinite power series 


= Tae 
k>0 
that converges for z = zo. Then the sum of absolute values 7,59 |ax2*| also 
converges whenever |z| < |zo|. If zo 4 0, we can therefore always write 


g(z) = ao + az +++ + amz” + O(2™T"). (11) 


For we have g(z) = ag + a12 +: +am2z™ +2™t!(am+1 +am4+22 +); we need 
only show that the parenthesized quantity is bounded when |z| < r, for some 
positive r, and it is easy to see that |am4i| + |@m+2|r + |am+3|r? +-+- is an 
upper bound whenever |z| < r < |zol. 

For example, the generating functions listed in Section 1.2.9 give us many 
important asymptotic formulas valid when z is sufficiently small, including 


1 1 
e= 1 Hz+ a2 te + 2™4 O(2""), (12) 
2! m! 
1 —1 m+1 
In(l+z) =z Ped = z™ + O(2™""), (13) 
2 m 
Q+) =1taz+ (g) ++ (S e+ 0G"), (14) 
m 
1 1 2 m m+1 
l =z + Hoe? +- + Hm” + O(2™tt), (15) 


1-z i l1-z 
for all nonnegative integers m. It is important to note that the hidden constants 
M and r implied by any particular O are related to each other. For example, 
the function e? is obviously O(1) when |z| < r, for any fixed r, since |e”| < e!?!; 
but there is no constant M such that |e*| < M for all values of z. Therefore we 
need to use larger and larger bounds M as the range r increases. 

Sometimes an asymptotic series is correct although it does not correspond 
to a convergent infinite series. For example, the basic formulas that express 
factorial powers in terms of ordinary powers, 

m 
=S e + oar, (16) 


k=0 
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are asymptotically valid for any real r and any fixed integer m > 0, yet the sum 
co 

y | 1/2 Jae 

= 1/2 —k 

diverges for all n. (See exercise 12.) Of course, when r is a nonnegative integer, 

n” and n” are simply polynomials of degree r, and (17) is essentially the same 

as 1.2.6-(44). When r is a negative integer and |n| > |r|, the infinite sum 
0 Lp pn)” does converge to n? = 1/(n—1)=; this sum can also be written 

in the more natural form X go finik, using Eq. 1.2.6-(58). 

Let us give one simple example of the concepts we have introduced so far. 
Consider the quantity /n; as n gets large, the operation of taking an nth root 
tends to decrease the value, but it is not immediately obvious whether ~/n 
decreases or increases. It turns out that {/n decreases to unity. Let us consider 
the slightly more complicated quantity n(¥/n — 1). Now ({/n — 1) gets smaller 
as n gets bigger; what happens to n(ņ\/n — 1)? 

This problem is easily solved by applying the formulas above. We have 


n= is aes (Inn/n) + O((Inn/n)*), (18) 


because ln n/n — 0 as n — oo; see exercises 8 and 11. This equation proves our 
previous contention that Vn — 1. Furthermore, it tells us that 


n(V/n — 1) =n(Inn/n + O((Inn/n)”)) = Inn + O((Inn)*/n). (19) 


In other words, n(V/n — 1) is approximately equal to Inn; the difference is 
O((In n)?/ n), which approaches zero as n approaches infinity. 


People often abuse O-notation by assuming that it gives an exact order of 
growth; they use it as if it specifies a lower bound as well as an upper bound. For 
example, an algorithm to sort n numbers might be called inefficient “because its 
running time is O(n?).” But a running time of O(n”) does not necessarily imply 
that the running time is not also O(n). There’s another notation, Big Omega, 
for lower bounds: The statement 


g(n) = 2(f(n)) (20) 
means that there are positive constants L and no such that 
|g(n)| > L|f(n)| for all n > no. 


Using this notation we can correctly conclude that a sorting algorithm whose 
running time is R(n?) will not be as efficient as one whose running time is 
O(nlogn), if n is large enough. However, without knowing the constant factors 
implied by O and 2, we cannot say anything about how large n must be before 
the O(nlogn) method will begin to win. 

Finally, if we want to state an exact order of growth without being precise 
about constant factors, we can use Big Theta: 


g(n) =O(f(n)) =  g(n)=0(f(n)) and g(n) = Q(f(n)). (21) 
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EXERCISES 
1. [HM01] What is limno O(n")? 


> 2. [M10] Mr. B. C. Dull obtained astonishing results by using the “self-evident” 


formula O(f(n)) — O(f(n)) = 0. What was his mistake, and what should the right- 
hand side of his formula have been? 


3. [M15] Multiply (nn+~7+O(1/n)) by (n+ O(./n)), and express your answer in 
O-notation. 


> 4. [M15] Give an asymptotic expansion of n(</a— 1), if a > 0, to terms O(1/n?). 


5. [M20] Prove or disprove: O(f(n)+ g(n)) = f(n) + O(g(n)), if f(n) and g(n) are 
positive for all n. (Compare with (10).) 


> 6. [M20] What is wrong with the following argument? “Since n = O(n), and 2n = 
O(n), ..., we have 


5 kn = 5 O(n) = O(n?).” 


7. [HM15] Prove that if m is any integer, there is no M such that e” < Mx” for 
arbitrarily large values of x. 


8. [HM20] Prove that as n > oo, (Inn)™/n > 0. 
9. [HM20]| Show that e°%”) = 1+ O(z™), for all fixed m > 0. 
10. [HM22] Make a statement similar to that in exercise 9 about In(1 + O(z”)). 
> 11. [M11] Explain why Eq. (18) is true. 


12. |HM25]| Prove that fire dl n—* does not approach zero as k — oo for any integer n, 


using the fact that Bee = (—4)* [z*] (ze*/(e* — 1))'”. 


> 13. [M10] Prove or disprove: g(n) = 2(f(n)) if and only if f(n) = O(g(n)). 


*1.2.11.2. Euler’s summation formula. One of the most useful ways to obtain 
good approximations to a sum is an approach due to Leonhard Euler. His method 
approximates a finite sum by an integral, and gives us a means to get better 
and better approximations in many cases. [Commentarii Academiz Scientiarum 
Imperialis Petropolitanze 6 (1732), 68-97.] 


1 2 3 4 5 6 7 
Fig. 12. Comparing a sum with an integral. 
Figure 12 shows a comparison of f} f(x) dx and YẸ] f(k), when n = 7. 


Euler’s strategy leads to a useful formula for the difference between these two 
quantities, assuming that f(x) is a differentiable function. 
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For convenience we shall use the notation 
{z} =xmodl=2-|2|. (1) 


Our derivation starts with the following identity: 


k+1 a k+1 
f ({} - 2) f'(a) de = (æ -k - 1) fæ) - f f(a) dz 
k k 


(This follows from integration by parts.) Adding both sides of this equation for 
1< k< n, we find that 


f Tairas 3S se = f " f(a) de: 
that is, 
E y= [sear s0) + [BUD I wea, (8 
1l<k<n 1 1 


where B;(x) is the polynomial x— 4. This is the desired connection between the 
sum and the integral. 

The approximation can be carried further if we continue to integrate by 
parts. Before doing this, however, we shall discuss the Bernoulli numbers, which 
are the coefficients in the following infinite series: 


z B22? By2* 
= Bot Bizt =+ = yo. (4) 


The coefficients of this series, which occur in a wide variety of problems, were 

introduced to European mathematicians in James Bernoulli’s Ars Conjectandi, 

published posthumously in 1713. Curiously, they were also discovered at about 

the same time by Takakazu Seki in Japan — and first published in 1712, shortly 

after his death. [See Takakazu Seki’s Collected Works (Osaka: 1974), 39—42.] 
We have 


further values appear in Appendix A. Since 


Z ee a ze *+l1 
ez—1— 


2 2e7-1 £2e-*%-1 
is an even function, we see that 


B; = Bs = By = Bg =--: =0. (6) 
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If we multiply both sides of the defining equation (4) by e* — 1, and equate 
coefficients of equal powers of z, we obtain the formula 


n 
S (7) Be = Bu + fn (7) 
k 
(See exercise 1.) We now define the Bernoulli polynomial 
By (x) = 2 y Bya™*. (8) 


If m = 1, then B(x) = Box + By = x — Ł, corresponding to the polynomial 
used above in Eq. (3). If m > 1, we have Bm(1) = Bm = Bm(0), by (7); in other 
words, Bm({x}) has no discontinuities at integer points x. 

The relevance of Bernoulli polynomials and Bernoulli numbers to our prob- 


lem will soon be clear. We find by differentiating Eq. (8) that 
Bi,(2) = D(F (m - k) Ba 


k 


m— 1 ; 
= m>>( k ) Bean 


k 
= mBm-1(2), (9) 


and therefore when m > 1, we can integrate by parts as follows: 


= [ Bra ({2}) f(a) dx = main? m+i(l ial )- Bri (0) f° (1) 
- tap ff Bees lap sa) ae. 


From this result we can continue to improve the approximation, Eq. (3), and we 
obtain Euler’s general formula: 


y= f fa) ae — 540) - $0) + FRM) - 0) + 
28 : 
i u i (P(n) — f8) + Rmn 
_ n Hod y á (f°) (n) fe D(1)) + Rmn, (10) 
1 k=1 “ 
using (6), where 
Ran = CEC S" Bal fa} sa) da. (13) 


The remainder Rmn will be small when By ({a})fo™(a)/m! is very small, and 
in fact, one can show that 
Bm({})| — |Bm] . _ 4 
m! 


<P < Gam (12) 
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when m is even. [See CMath, §9.5.] On the other hand, it usually turns out that 
the magnitude of f™ (x) gets large as m increases, so there is a “best” value of 
m at which |Rmn| has its least value when n is given. 

It is known that, when m is even, there is a number 0 such that 


Ran = OPE (FON (n) — FOC), O<O<1, Ga) 
provided that f+?) (x) f("+4 (x) > 0 for 1 < x <n. So in these circumstances 
the remainder has the same sign as, and is less in absolute value than, the first 
discarded term. A simpler version of this result is proved in exercise 3. 

Let us now apply Euler’s formula to some important examples. First, we 
set f(x) = 1/x. The derivatives are f™ (£) = (—1)™m!/x2™*1, so we have, by 
Eq. (10), 


Bk k-ı/ 1l 
Hn BE Ta. (= 1) + Rmn- (14) 
Now we find 
ee) 
y= lim (A,_1 — Inn) a ae Roy Jim Rays (15) 
n= oo T 
The fact that inns Run = — JE B m({x}) dx/a™* exists proves that the 


constant y does in fact exist. We can o put Eqs. (14) and (15) together, 
to deduce a general approximation for the harmonic numbers: 


Me ‘an 1 
Ay-1 =Inn+ KT yo n Pelio) dr 
k=1 
m—1 
(—1)*-1B, 1 
=Inn+ y+ ; 
Bia ee a sold) 
k=1 
Replacing m by m + 1 yields 
m = 1B, 1 
Hp- peinntyt OO Tok +0(—,). (16) 


Furthermore, by Eq. (13) we see that the error is less than the first term 
discarded. As a particular case we have (adding 1/n to both sides) 


1 1 1 reent 1 
t E; € —— = 
2n 12n? = 120n4 


A, =1 | ‘ 
Peet 6n®  252n° 

This is Eq. 1.2.7-(3). The Bernoulli numbers Bẹ for large k get very large 

(approximately (—1)!*+*/?2(k!/(2m)*) when k is even), so Eq. (16) cannot be 

extended to a convergent infinite series for any fixed value of n. 
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The same technique can be applied to deduce Stirling’s approximation. This 
time we set f(x) = Ina, and Eq. (10) yields 


l l B,(-1)¥ ¢ 1 
In(n — 1)! =ninn-n+1-$lnn4 5 Ek 1) (== 1) H Rmn. (17) 
1<k<m 
Proceeding as above, we find that the limit 


1)¥+1 
lim (Inn! —nlnn+n-— Inn) y= 1+ > +t + lim Rmn 
noo 


n— o0 
l<k<m 


exists; let it be called o (“Stirling’s constant”) temporarily. We get Stirling’s 


result jt i 
Inn! = (n+ 5)Inn o + So ik a I -o(—). (18) 


l<k<m 


In particular, let m = 5; we have 


1 1 1 
E TE ( ). 
ee eel ee ad 


Now we can take the exponential of both sides: 


s(n? 1 1 afl 
a va(Z) ep( z 3607? -0(5)). 


Using the fact that e7 = 27 (see exercise 5), and expanding the exponential, 
we get our final result: 


J- Im (2Y 7 1o 1 139 571 a) (19) 
S e ' 12n ` 288n2 51840n3 2488320n4 ` n5) J Ne 


EXERCISES 

1. [M18] Prove Eq. (7). 

2. [HM20] Note that Eq. (9) follows from Eq. (8) for any sequence Bn, not only for 
the sequence defined by Eq. (4). Explain why the latter sequence is necessary for the 
validity of Eq. (10). 

3. [HM20] Let Cmn = (Bm/M!) (fY (n) — f°") (1)) be the mth correction term 
in Euler’s summation formula. Assuming that f (m) (x) has a constant sign for all x in 
the range 1 < x < n, prove that |Rmn| < |Cmn| when m = 2k > 0; in other words, 
show that the remainder is not larger in absolute value than the last term computed. 


4. [HM20] (Sums of powers.) When f(x) = x”, the high-order derivatives of f are 
all zero, so Euler’s summation formula gives an exact value for the sum 


Le 


O<k<n 


in terms of Bernoulli numbers. (It was the study of Sm(n) for m = 1, 2, 3, ... that 
led Bernoulli and Seki to discover those numbers in the first place.) Express Sm(n) in 
terms of Bernoulli polynomials. Check your answer for m = 0, 1, and 2. (Note that 
the desired sum is performed for 0 < k < n instead of 1 < k < n; Euler’s summation 
formula may be applied with 0 replacing 1 throughout.) 
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5. [HM30] Given that 


nt=nva(2)'(1+0(2)), 


show that x = /27 by using Wallis’s product (exercise 1.2.5-18). [Hint: Consider (*”) 
for large values of n.] 


> 6. [HM30] Show that Stirling’s approximation holds for noninteger n as well: 


r(x+1)= oma (2) (2 H o(3)): z>a>0. 


[Hint: Let f(x) = ln(x +c) in Euler’s summation formula, and apply the definition of 
I(x) given in Section 1.2.5.] 
> 7. [HM32] What is the approximate value of 112733...n”? 

8. [M23] Find the asymptotic value of In (an?+bn)! with absolute error O(n~"). Use 
it to compute the asymptotic value of (e2?) /(c” (”)) with relative error O(n~7), when 
cis a positive constant. Here absolute error e means that (truth) = (approximation) +€; 
relative error € means that (truth) = (approximation) (1+ €). 

> 9. [M25] Find the asymptotic value of (22) with a relative error of O(n~3), in two 
ways: (a) via Stirling’s approximation; (b) via exercise 1.2.6-47 and Eq. 1.2.11.1—(16). 


*1.2.11.3. Some asymptotic calculations. In this subsection we shall inves- 
tigate the following three intriguing sums, in order to deduce their approximate 
values: 


—1 =. =D “\ (n— k)*(n — k)! 
Piste +42 PE ai oes tis eles) (1) 
n n n-li n! 

k=0 
n—1 n-ln-2 2 n! 
URET E er Re (2) 
k=1 
R(n) =1 n n n ni nk (3) 
oo oe i eae oe 
n+1 n+1n+2 Ss (n+ ky)! ° 


These functions, which are similar in appearance yet intrinsically different, arise 
in several algorithms that we shall encounter later. Both P(n) and Q(n) are 
finite sums, while R(n) is an infinite sum. It seems that when n is large, all 
three sums will be nearly equal, although it is not obvious what the approximate 
value of any of them will be. Our quest for approximate values of these functions 
will lead us through a number of very instructive side results. (You may wish to 
stop reading temporarily and try your hand at studying these functions before 
going on to see how they are attacked here.) 
First, we observe an important connection between Q(n) and R(n): 


Q(n) + R(n) - (( n+- + wt ("+ wt) 


n” (n-1 n! (n+1 


n!e” 


Z , (4) 


n” 
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Stirling’s formula tells us that n!e"/n” is approximately /27n, so we can guess 
that Q(n) and R(n) will each turn out to be roughly equal to ,/7n/2. 

To get any further we must consider the partial sums of the series for e”. 
By using Taylor’s formula with remainder, 
F™ 0)” 


T 
t” 


rf Freed, (5) 
o n! 


we are soon led to an important function known as the incomplete gamma 
function: 


yla, x£) = n ee dt. (6) 


We shall assume that a > 0. By exercise 1.2.5-20, we have (a, œ0) = I (a); this 
accounts for the name “incomplete gamma function.” It has two useful series 
expansions in powers of x (see exercises 2 and 3): 


ze zetl get? (—1)¥akte 
mans atl + Aara 2 kl (k Fa) ’ (7) 
xe etl get? gkta 
Pens aaa) aare DTE (8) 
From the second formula we see the connection with R(n): 
onle ( y(n,n) 
Rn) == (Ae). (9) 


This equation has purposely been written in a more complicated form than 
necessary, since y(n, n) is a fraction of y(n, o0) = I (n) = (n — 1)!, and n! e"/n” 
is the quantity in (4). 

The problem boils down to getting good estimates of y(n,n)/(n — 1)!. We 
shall now determine the approximate value of y(x +1, «+ y)/I'(a+1), when y 
is fixed and z is large. The methods to be used here are more important than 
the results, so the reader should study the following derivation carefully. 

By definition, we have 

x+y 
ya g 1, z+ y) 1 j if ett dt 
0 


F(e+1) = P(e+1 


fe T a app — ee Sadik (10) 
=1— s 
Tathl, © Teth/, ~ e 


co 

n= f e™t dt, 
x 
x+y 

n= | e ‘t® dt, 
x 


and consider each integral in turn. 


Let us set 
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Estimate of Iı: We convert J, to an integral from 0 to infinity by substituting 
t = (1+ u); we further substitute v = u—In(1 + u), dv = (1 — 1/(1 + u)) du, 
which is legitimate since v is a monotone function of u: 


co co 1 
I= eta? f ze *"(1+u)"* du = a. re 7” (1 + =) dv. (11) 
0 0 u 
In the last integral we will replace 1 + 1/u by a power series in v. We have 


v= dy? — tu’ 4 jut zu pees = (u?/2)(1 zu u? 2u’ Hee. 


1 
2 


Setting w = /2v, we have therefore 


L2 
=U zu 4 


24 lu? 2y34...)1/2 


7,3 73,4 1331 „5 6 
3 U r 95 5 T O(u ). 


w=u(l 36% — 540% T T2960% 7 

(This expansion may be obtained by the binomial theorem; efficient methods for 
performing such transformations, and for doing the other power series manipu- 
lations needed below, are considered in Section 4.7.) We can now solve for u as 
a power series in w: 


là 


2 1 2 1 3 1 5 6). 

ea ta org aaa POM): 
111 2 1 

va a go ee ee 


1 u2, 2 V2 np 4 v2 a, 
12 135 ` 432 l 


In all of these formulas, the O-notation refers to small values of the argu- 
ment, that is, |u| < r, |v] < r, |w| < r for sufficiently small positive r. Is this 
good enough? The substitution of 1 + 1/u in terms of v in Eq. (11) is supposed 
to be valid for 0 < v < œ, not only for |v| < r. Fortunately, it turns out that 
the value of the integral from 0 to oo depends almost entirely on the values of 
the integrand near zero. In fact, we have (see exercise 4) 


a cer (1 + ~) dv = O(e "”) (13) 


for any fixed r > 0 and for large x. We are interested in an approximation up 
to terms O(x~™), and since O((1/e")*) is much smaller than O(2~™) for any 
positive r and m, we need integrate only from 0 to r, for any fixed positive r. We 
therefore take r to be small enough so that all the power series manipulations 
done above are justified (see Eqs. 1.2.11.1-(11) and 1.2.11.3-(13)). 

Now 


Co Co 1 
f zev" dv = =| e 4q*dq= —T(a+1), ifa>-—t, (14) 
0 0 T 


O(v?). (12) 


so by plugging the series (12) into the integral (11) we have finally 


T 2 V20 4 V 2T 
I = etr? y= 1/2 -1/2 =i -3/2 4 O(x7? ). 
1=e "2 ( z7 tzt 4 2 135” tE” +O(a~“)}). (15) 
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Estimate of I>: In the integral Iz, we substitute t = u + x and obtain 


y uN? 
h= | e“(1+ =) du. (16) 
0 x 
Now 
(142) =e +21 (+4) = ex (H+ H+ 0 a 
e z) Pere a)) “P\ on * 322 ? 
2 4 3 
—]_ u u +0(27°) 


for 0 < u < y and large x. Therefore we find that 


y a (E N 3 
h =e "x" |y- a —+ —)a-*4+ Olx’) ). 
25er (v g +(5+5)e + O(a )) (17) 

Finally, we analyze the coefficient e~*%*/I'(a + 1) that appears when we 
multiply Eqs. (15) and (17) by the factor 1/f’(a + 1) in (10). By Stirling’s 
approximation, which is valid for the gamma function by exercise 1.2.11.2—6, we 
have 


x e7 1/121+0(27*) 


ees, 1 3/2 1 -5/2 -7/2 
T x x + O(a . (18 
V2 12/27 288V 2T ( yee) 


And now the grand summing up: Equations (10), (15), (17), and (18) yield 


Theorem A. For large values of x, and fixed y, 


y(r +1, x+y) _i y — 2/3 go l/2 4 1 (= y PY a 
T(z +1) 2 fy ' 2n \270 12 6 
+O(a-*/?), | (19) 


The method we have used shows how this approximation could be extended to 
further powers of x as far as we please. 

Theorem A can be used to obtain the approximate values of R(n) and Q(n), 
by using Eqs. (4) and (9), but we shall defer that calculation until later. Let us 
now turn to P(n), for which somewhat different methods seem to be required. 
We have 


n—k 
P(n) a5" ee al (14 pe + OCF )). (20) 


k=0 


Thus to get the values of P(n), we must study sums of the form 


n 


5 kT Ie E, 


k=0 
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Let f(x) = 2"*1/2e-* and apply Euler’s summation formula: 


n n 
> krtl/2e—k — J r” tl/e-® dy + e + Ante” — R. (21) 
0 

k=0 


A crude analysis of the remainder (see exercise 5) shows that R = O(n"e7"); 
and since the integral is an incomplete gamma function, we have 
n 
5 krtl/2e—k — y(n + 3, n) + re. +O(n"e"). (22) 
k=0 


Our formula, Eq. (20), also requires an estimate of the sum 


n 
yee t= 5 k” 1)+1/2, kyn” 1/2, a 
k=0 0<k<n—-1 


and this can also be obtained by Eq. (22). 

We now have enough formulas at our disposal to determine the approximate 
values of P(n), Q(n), and R(n), and it is only a matter of substituting and 
multiplying, etc. In this process we shall have occasion to use the expansion 


(nta = nte (1 $ a(s = <) L n ow-)) , (23) 


which is proved in exercise 6. The method of (21) yields only the first two terms 
in the asymptotic series for P(n); further terms can be obtained by using the 
instructive technique described in exercise 14. 

The result of all these calculations gives us the desired asymptotic formulas: 


m 2 1l jr 4 71 T _9 
Pm=V>-3t og Von.’ 135m. 1150) Ine OE Ge 
mn 1 1 T 4 1 T _9 
1”) => - 3+ DV mn” Ten t 288 V 2n3 * OWE); (25) 


mn 1 1 T 4 1 T _9 
EM) 41 +3 Poa Ten pee ee tO A 


The functions studied here have received only light treatment in the pub- 
lished literature. The first term \/an/2 in the expansion of P(n) was given by 
H. B. Demuth [Ph.D. thesis (Stanford University, October 1956), 67-68]. Using 
this result, a table of P(n) for n < 2000, and a good slide rule, the author 
proceeded in 1963 to deduce the empirical estimate P(n) ~ ,/an/2 — 0.6667 + 
0.575/,/n. It was natural to conjecture that 0.6667 was really an approximation 
to 2/3, and that 0.575 would perhaps turn out to be an approximation to 
y = 0.57721... (why not be optimistic?). Later, as this section was being 
written, the correct expansion of P(n) was developed, and the conjecture 2/3 
was verified; for the other coefficient 0.575 we have not y but H y T/2 ~ 0.5744. 
This nicely confirms both the theory and the empirical estimates. 
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Formulas equivalent to the asymptotic values of Q(n) and R(n) were first 
determined by the brilliant self-taught Indian mathematician S. Ramanujan, 
who posed the problem of estimating n! e"/2n” — Q(n) in J. Indian Math. Soc. 
3 (1911), 128; 4 (1912), 151-152. In his answer to the problem, he gave the 
asymptotic series f } iz nt 5 72 s n73 +- , which goes considerably 
beyond Eq. (25). His derivation was somewhat more elegant than the method 
described above; to estimate I,, he substituted t = x + u/2z, and expressed 
the integrand as a sum of terms of the form cjg i exp(—u?)ua—*/? du. The 
integral Iz can be avoided completely, since ay(a, x) = e%e~* + y(a +1, x) when 
a > 0; see (8). An even simpler approach to the asymptotics of Q(n), perhaps 
the simplest possible, appears in exercise 20. The derivation we have used, 
which is instructive in spite of its unnecessary complications, is due to R. Furch 
[Zeitschrift für Physik 112 (1939), 92-95], who was primarily interested in the 
value of y that makes y(x + 1, x + y) = I(x + 1)/2. The asymptotic properties 
of the incomplete gamma function were later extended to complex arguments by 
F. G. Tricomi [Math. Zeitschrift 53 (1950), 136-148]. See also N. M. Temme, 
Math. Comp. 29 (1975), 1109-1114; SIAM J. Math. Anal. 10 (1979), 757-766. 
H. W. Gould has listed references to several other investigations of Q(n) in AMM 
75 (1968), 1019-1021. 

Our derivations of the asymptotic series for P(n), Q(n), and R(n) use only 
simple techniques of elementary calculus; notice that we have used different 
methods for each function! Actually we could have solved all three problems 
using the techniques of exercise 14, which are explained further in Sections 5.1.4 
and 5.2.2. That would have been more elegant but less instructive. 

For additional information, interested readers should consult the beautiful 
book Asymptotic Methods in Analysis by N. G. de Bruijn (Amsterdam: North- 
Holland, 1958). See also the more recent survey by A. M. Odlyzko [Handbook 
of Combinatorics 2 (MIT Press, 1995), 1063-1229], which includes 65 detailed 
examples and an extensive bibliography. 


EXERCISES 
1. [HM20] Prove Eq. (5) by induction on n. 
2. [HM20] Obtain Eq. (7) from Eq. (6). 
3. [M20] Derive Eq. (8) from Eq. (7). 
> 4. [HM10] Prove Eq. (13). 
5. [HM24] Show that R in Eq. (21) is O(n"e~"”). 
> 6. [HM20] Prove Eq. (23). 
> 7. |HM30| In the evaluation of I2, we had to consider f e” (1 + ey du. Give an 
0 


asymptotic representation of 


to terms of order O(a~*), when y is fixed and 2 is large. 


122 BASIC CONCEPTS 1.2.11.3 


8. [HM30] If f(a) = O(a") as x > œ and 0 < r < 1, show that 


f(x) u Ur? f(x) —u? u3 —1 m=1,m p 
I e (1+) a= f exp ( Aa = aan ( T ) du + O(a °) 
ifm = [( /—1)]. [This proves in particular a result due to Tricomi: If 


f(z) = an 
i av“ =) du = vie [Oe dt + O(1).] 


> 9. [HM36] What is the behavior of y(x + 1, pxr)/I'(x + 1) for large x? (Here p is a 
real constant; and if p < 0, we assume that x is an integer, so that t” is well defined for 
negative t.) Obtain at least two terms of the asymptotic expansion, before resorting 
to O-terms. 


10. [HM34] Under the assumptions of the preceding problem, with p 4 1, obtain the 
asymptotic expansion of y(x +1, px + py/(p—1)) — y(x +1, px), for fixed y, to terms 
of the same order as obtained in the previous exercise. 


> 11. [HM35] Let us generalize the functions Q(n) and R(n) by introducing a parame- 


ter z: 
n—-1 n-1ln-2 > 
£ £ 


n n n 

n n n 9 
n+1 S n+1in+2 . 
Explore this situation and find asymptotic formulas when x # 1. 


Rz(n) = 1 


12. [HM20] The function fhe ? e™t?/2 dt that appeared in connection with the normal 
distribution (see Section 1.2. 10) can be expressed as a special case of the incomplete 
gamma function. Find values of a, b, and y such that by(a, y) o in © T et)? dt, 
13. [HM42] (S. Ramanujan.) Prove that R(n) — Q(n) = 2 + 8/(135(n + A(n))), 
where 4 < 0(n) < #. (This implies the much weaker result R(n + 1) — Q(n + i < 
R(n) — Q(n).) 
> 14. [HM39] (N. G. de Bruijn.) The purpose of this exercise is to find the asymptotic 
expansion of )77_, k”t%e~* for fixed a, as n — oo. 
a) Replacing k by n—k, show that the given sum equals n"*°e~” 5 po en F/2m Ff, n), 
where 


f(k,n) = (1-=)"en(--G---). 


b) Show that for all m > 0 and e > 0, the quantity f(k,m) can be written in the form 


cik’ tin ii 4 O(n +DO1239) if 0< k< nt, 


O<i<jgm 
c) Prove that as a consequence of (b), we have 
2 2 2 
No e™ /2m Fk, n) =. y cagn 9 DD kti e k 2n ae On 9"), 
k=0 0<i<j<m k>0 


for all 5 > 0. [Hint: The sums over the range n'/?+* < k < oo are O(n~") for 
all r.] 
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d) Show that the asymptotic expansion of $ >o kte-*’/2" for fixed t > 0 can be 
obtained by Euler’s summation formula. 7 


e) Finally therefore 


n 


1 


TN 


-d 


D 


Ha = n tte?” ( 
k=0 


2 


this computation can in principle be 


T 


extended to O(n~") for any desired r. 


12 2 


15. [HM20] Show that the following integral is related to Q(n): 


fat: 


16. [M24] Prove the identity 


ey (ew) 


k 


n 
k 


=) e dz. 


(-1)"(n—1)!,. when n> 0. 


17. [HM29] (K. W. Miller.) Symmetry demands that we consider also a fourth series, 


which is to P(n) as R(n) is to Q(n): 


j n j n Ns 


S(n)=1 


n+1 n+2 n4 


What is the asymptotic behavior of this function? 


18. [M25] Show that the sums > (%) k” 


n= k)” and X (")(k+1)'(n—k)"* can 


be expressed very simply in terms of the Q function. 


19. [HM30] (Watson’s lemma.) Show that if the integral Cn = 


0 f(z) dg exists 


for all large n, and if f(x) = O(a) for 0 < x < r, where r > 0 and a > —1, then 


Cn = O(n ®). 
> 20. [HM30] Let u = w4 iw? H aw sw ++++= JO? chw” be the power series 
solution to the equation w = (u? — $u + fut — ĉu” 4 --+)*/?. as in (12). Show that 


m—1 


Q(n) +1= So keg (k 


k=1 


for all m > 1. [Hint: Apply Watson’s lem 


ID (Z) + ofa?) 


ma to the identity of exercise 15.] 


| feel as if | should succeed in doing something in mathematics, 
although I cannot see why it is so very important. 


— HELEN KELLER (1898) 
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1.3. MIX 


IN MANY PLACES throughout this book we will have occasion to refer to a com- 
puter’s internal machine language. The machine we use is a mythical computer 
called “MIX.” MIX is very much like nearly every computer of the 1960s and 
1970s, except that it is, perhaps, nicer. The language of MIX has been designed 
to be powerful enough to allow brief programs to be written for most algorithms, 
yet simple enough so that its operations are easily learned. 

The reader is urged to study this section carefully, since MIX language 
appears in so many parts of this book. There should be no hesitation about 
learning a machine language; indeed, the author once found it not uncommon to 
be writing programs in a half dozen different machine languages during the same 
week! Everyone with more than a casual interest in computers will probably get 
to know at least one machine language sooner or later. MIX has been specially 
designed to preserve the simplest aspects of historic computers, so that its 
characteristics are easy to assimilate. 


D However, it must be admitted that MIX is now quite obsolete. Therefore 
1 MIX will be replaced in subsequent editions of this book by a new machine 
called MMIX, the 2009. MMIX will be a so-called reduced instruction set computer 
(RISC), which will do arithmetic on 64-bit words. It will be even nicer than MIX, 
and it will be similar to machines that have become dominant during the 1990s. 
The task of converting everything in this book from MIX to MMIX will take a 
long time; volunteers are solicited to help with that conversion process. Mean- 
while, the author hopes that people will be content to live for a few more years 
with the old-fashioned MIX architecture — which is still worth knowing, because 
it helps to provide a context for subsequent developments. 


1.3.1. Description of MIX 


MIX is the world’s first polyunsaturated computer. Like most machines, it has 
an identifying number — the 1009. This number was found by taking 16 actual 
computers very similar to MIX and on which MIX could easily be simulated, then 
averaging their numbers with equal weight: 


| (360 + 650 + 709 + 7070 + U3 + SS80 + 1107 + 1604 + G20 + B220 
+ S2000 + 920 + 601 + H800 + PDP-4 + II)/16] = 1009. (1) 


The same number may also be obtained in a simpler way by taking Roman 
numerals. 

MIX has a peculiar property in that it is both binary and decimal at the same 
time. MIX programmers don’t actually know whether they are programming a 
machine with base 2 or base 10 arithmetic. Therefore algorithms written in 
MIX can be used on either type of machine with little change, and MIX can be 
simulated easily on either type of machine. Programmers who are accustomed 
to a binary machine can think of MIX as binary; those accustomed to decimal 
may regard MIX as decimal. Programmers from another planet might choose to 
think of MIX as a ternary computer. 
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Words. The basic unit of MIX data is a byte. Each byte contains an unspecified 
amount of information, but it must be capable of holding at least 64 distinct 
values. That is, we know that any number between 0 and 63, inclusive, can be 
contained in one byte. Furthermore, each byte contains at most 100 distinct 
values. On a binary computer a byte must therefore be composed of six bits; on 
a decimal computer we have two digits per byte.* 

Programs expressed in MIX’s language should be written so that no more 
than sixty-four values are ever assumed for a byte. If we wish to treat the 
number 80, we should always leave two adjacent bytes for expressing it, even 
though one byte is sufficient on a decimal computer. An algorithm in MIX should 
work properly regardless of how big a byte is. Although it is quite possible to 
write programs that depend on the byte size, such actions are anathema to the 
spirit of this book; the only legitimate programs are those that would give correct 
results with all byte sizes. It is usually not hard to abide by this ground rule, 
and we will thereby find that programming a decimal computer isn’t so different 
from programming a binary one after all. 

Two adjacent bytes can express the numbers 0 through 4,095. 

Three adjacent bytes can express the numbers 0 through 262,143. 

Four adjacent bytes can express the numbers 0 through 16,777,215. 

Five adjacent bytes can express the numbers 0 through 1,073,741,823. 

A computer word consists of five bytes and a sign. The sign portion has 
only two possible values, + and —. 


Registers. There are nine registers in MIX (see Fig. 13): 


The A-register (Accumulator) consists of five bytes and a sign. 

The X-register (Extension), likewise, comprises five bytes and a sign. 

The I-registers (Index registers) I1, I2, I3, 14, 15, and I6 each hold two bytes 
together with a sign. 

The J-register (Jump address) holds two bytes; it behaves as if its sign is 
always +. 


We shall use a small letter “r”, prefixed to the name, to identify a MIX register. 
Thus, “rA” means “register A.” 

The A-register has many uses, especially for arithmetic and for operating 
on data. The X-register is an extension on the “right-hand side” of rA, and it 
is used in connection with rA to hold ten bytes of a product or dividend, or 
it can be used to hold information shifted to the right out of rA. The index 
registers rI1, rI2, rI3, rl4, rI5, and rI6 are used primarily for counting and for 
referencing variable memory addresses. The J-register always holds the address 
of the instruction following the most recent “jump” operation, and it is primarily 
used in connection with subroutines. 


* Since 1975 or so, the word “byte” has come to mean a sequence of precisely eight binary 
digits, capable of representing the numbers 0 to 255. Real-world bytes are therefore larger than 
the bytes of the hypothetical MIX machine; indeed, MIX’s old-style bytes are just barely bigger 
than nybbles. When we speak of bytes in connection with MIX we shall confine ourselves to 
the former sense of the word, harking back to the days when bytes were not yet standardized. 
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Fig. 13. The MIX computer. 
Besides its registers, MIX contains 
an overflow toggle (a single bit that is either “on” or “off”); 
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a comparison indicator (having three values: LESS, EQUAL, or GREATER); 
memory (4000 words of storage, each word with five bytes and a sign); 
and input-output devices (cards, tapes, disks, etc.). 


Partial fields of words. The five bytes and sign of a computer word are 


numbered as follows: 


+ | Byte | Byte 


Byte 


Byte 


Byte 


(2) 
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Most of the instructions allow a programmer to use only part of a word if desired. 
In such cases a nonstandard “field specification” can be given. The allowable 
fields are those that are adjacent in a computer word, and they are represented 
by (L:R), where L is the number of the left-hand part and R is the number of 
the right-hand part of the field. Examples of field specifications are: 

(0:0), the sign only. 

(0:2), the sign and the first two bytes. 

(0:5), the whole word; this is the most common field specification. 

(1:5), the whole word except for the sign. 

(4:4), the fourth byte only. 

(4:5), the two least significant bytes. 


The use of field specifications varies slightly from instruction to instruction, 
and it will be explained in detail for each instruction where it applies. Each 
field specification (L:R) is actually represented inside the machine by the single 
number 8L + R; notice that this number fits easily in one byte. 


Instruction format. Computer words used for instructions have the following 
form: 


0 1 2 3 4 5 


(3) 
+ A A L ITITFI C 


The rightmost byte, C, is the operation code telling what operation is to 
be performed. For example, C = 8 specifies the operation LDA, “load the A- 
register.” 

The F-byte holds a modification of the operation code. It is usually a field 
specification (L:R) = 8L + R; for example, if C = 8 and F = 11, the operation 
is “load the A-register with the (1:3) field.” Sometimes F is used for other 
purposes; on input-output instructions, for example, F is the number of the 
relevant input or output unit. 

The left-hand portion of the instruction, +AA, is the address. (Notice that 
the sign is part of the address.) The I-field, which comes next to the address, 
is the index specification, which may be used to modify the effective address. 
If I = 0, the address +AA is used without change; otherwise I should contain 
a number i between 1 and 6, and the contents of index register Ii are added 
algebraically to +AA before the instruction is carried out; the result is used as 
the address. This indexing process takes place on every instruction. We will use 
the letter M to indicate the address after any specified indexing has occurred. 
(If the addition of the index register to the address +AA yields a result that 
does not fit in two bytes, the value of M is undefined.) 

In most instructions, M will refer to a memory cell. The terms “memory 
cell” and “memory location” are used almost interchangeably in this book. We 
assume that there are 4000 memory cells, numbered from 0 to 3999; hence every 
memory location can be addressed with two bytes. For every instruction in which 
M refers to a memory cell we must have 0 < M < 3999, and in this case we will 
write CONTENTS (M) to denote the value stored in memory location M. 


128 BASIC CONCEPTS 1.3.1 


On certain instructions, the “address” M has another significance, and it 
may even be negative. Thus, one instruction adds M to an index register, and 
such an operation takes account of the sign of M. 


Notation. To discuss instructions in a readable manner, we will use the notation 
OP ADDRESS,I(F) (4) 


to denote an instruction like (3). Here OP is a symbolic name given to the 
operation code (the C-part) of the instruction; ADDRESS is the +AA portion; I 
and F represent the I- and F-fields, respectively. 

If I is zero, the ‘,1’ is omitted. If F is the normal F-specification for this 
particular operator, the ‘(F)’ need not be written. The normal F-specification 
for almost all operators is (0:5), representing a whole word. If a different F is 
normal, it will be mentioned explicitly when we discuss a particular operator. 

For example, the instruction to load a number into the accumulator is called 
LDA and it is operation code number 8. We have 


Conventional representation Actual numeric instruction 
LDA 2000,2(0:3) + 2000/2]3]8 
LDA 2000,2(1:3) + 2000| 211 8 
LDA 2000(1:3) + 2000 0 |11/ 8 (5) 
LDA 2000 + 2000/0|/5]8 
LDA -2000,4 - 2000} 4] 5/8 


The instruction ‘LDA 2000,2(0:3)’ may be read “Load A with the contents of 
location 2000 indexed by 2, the zero-three field.” 

To represent the numerical contents of a MIX word, we will always use a box 
notation like that above. Notice that in the word 


+ 2000/2/3|8 


the number +2000 is shown filling two adjacent bytes and sign; the actual 
contents of byte (1:1) and of byte (2:2) will vary from one MIX computer to 
another, since byte size is variable. As a further example of this notation for MIX 
words, the diagram 


= 10000 |3000 


represents a word with two fields, a three-byte-plus-sign field containing —10000 
and a two-byte field containing 3000. When a word is split into more than one 
field, it is said to be “packed.” 


Rules for each instruction. The remarks following (3) above have defined 
the quantities M, F, and C for every word used as an instruction. We will now 
define the actions corresponding to each instruction. 
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Loading operators. 


e LDA (load A). C = 8; F = field. 
The specified field of CONTENTS(M) replaces the previous contents of register A. 

On all operations where a partial field is used as an input, the sign is used 
if it is a part of the field, otherwise the sign + is understood. The field is shifted 
over to the right-hand part of the register as it is loaded. 

Examples: If F is the normal field specification (0:5), everything in location 
M is copied into rA. If F is (1:5), the absolute value of CONTENTS (M) is loaded 
with a plus sign. If M contains an instruction word and if F is (0:2), the “+AA” 
field is loaded as 


+lololo A A 


Suppose location 2000 contains the word 


-| so J3ļls5sl4]; (6) 


then we get the following results from loading various partial fields: 


Instruction Contents of rA afterwards 
LDA 2000 -| so |3|5/4 
LDA 2000(1:5) +| 80 |3/5)4 
LDA 2000(3:5) +/0/0/3/5)4 
LDA 2000(0:3) -/0]/0} 80 |3 
LDA 2000(4:4) +/0/0/0/0/5 
LDA 2000(0:0) -/0}/0;0/;0}]0 
LDA 2000(1:1) +/0/0/0/0/7 


(The last example has a partially unknown effect, since byte size is variable.) 


e LDX (load X). C = 15; F = field. 
This is the same as LDA, except that rX is loaded instead of rA. 


e LDi (load i). C = 8 + i; F = field. 
This is the same as LDA, except that rIi is loaded instead of rA. An index register 
contains only two bytes (not five) and a sign; bytes 1, 2, 3 are always assumed 
to be zero. The LD? instruction is undefined if it would result in setting bytes 1, 
2, or 3 to anything but zero. 

In the description of all instructions, “i” stands for an integer, 1 <i < 6. 
Thus, LDi stands for six different instructions: LD1, LD2, ..., LD6. 
e LDAN (load A negative). C = 16; F = field. 
e LDXN (load X negative). C = 23; F = field. 
e LDiN (load i negative). C = 16 + i; F = field. 
These eight instructions are the same as LDA, LDX, LDi, respectively, except that 
the opposite sign is loaded. 
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Storing operators. 


e STA (store A). C = 24; F = field. 
A portion of the contents of rA replaces the field of CONTENTS (M) specified by F. 
The other parts of CONTENTS (M) are unchanged. 

On a store operation the field F has the opposite significance from the load 
operation: The number of bytes in the field is taken from the right-hand portion 
of the register and shifted left if necessary to be inserted in the proper field of 
CONTENTS(M). The sign is not altered unless it is part of the field. The contents 
of the register are not affected. 


Examples: Suppose that location 2000 contains 


-|}1/2/3/4]5 


and register A contains 


Then: 
Instruction Contents of location 2000 afterwards 
STA 2000 +/6|7/8/9]0 
STA 2000(1:5) -16/7/8900 
STA 2000(5:5) -1/2/3410 
STA 2000(2:2) -/1/0/3)4]5 
STA 2000(2:3) -/1)/9|0);4]5 
STA 2000(0:1) +/0)/2/3)/41]5 


e STX (store X). C = 31; F = field. 
Same as STA, except that rX is stored rather than rA. 
e STi (store i). C = 24+ i; F = field. 
Same as STA, except that rIi is stored rather than rA. Bytes 1, 2, 3 of an index 
register are zero; thus if rll contains 


EIMIN 


it behaves as though it were 


e STJ (store J). C = 32; F = field. 
Same as STi, except that rJ is stored and its sign is always +. 

With STJ the normal field specification for F is (0:2), not (0:5). This is 
natural, since STJ is almost always done into the address field of an instruction. 
e STZ (store zero). C = 33; F = field. 

Same as STA, except that plus zero is stored. In other words, the specified field 
of CONTENTS (M) is cleared to zero. 
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Arithmetic operators. On the add, subtract, multiply, and divide operations, 
a field specification is allowed. A field specification of “(0:6)” can be used to 
indicate a “floating point” operation (see Section 4.2), but few of the programs 
we will write for MIX will use this feature, since we will primarily be concerned 
with algorithms on integers. 

The standard field specification is, as usual, (0:5). Other fields are treated 
as in LDA. We will use the letter V to indicate the specified field of CONTENTS (M); 
thus, V is the value that would have been loaded into register A if the operation 
code were LDA. 


e ADD. C = 1; F = field. 
V is added to rA. If the magnitude of the result is too large for register A, the 
overflow toggle is set on, and the remainder of the addition appearing in rA is as 
though a “1” had been carried into another register to the left of rA. (Otherwise 
the setting of the overflow toggle is unchanged.) If the result is zero, the sign of 
rA is unchanged. 
Example: The sequence of instructions below computes the sum of the five 

bytes of register A. 

STA 2000 

LDA 2000(5:5) 

ADD 2000(4:4) 

ADD 2000(3:3) 

ADD 2000(2:2) 

ADD 2000(1:1) 


This is sometimes called “sideways addition.” 

Overflow will occur in some MIX computers when it would not occur in 
others, because of the variable definition of byte size. We have not said that 
overflow will occur definitely if the value is greater than 1073741823; overflow 
occurs when the magnitude of the result is greater than the contents of five bytes, 
depending on the byte size. One can still write programs that work properly and 
that give the same final answers, regardless of the byte size. 


e SUB (subtract). C = 2; F = field. 
V is subtracted from rA. (Equivalent to ADD but with —V in place of V.) 


e MUL (multiply). C = 3; F = field. 

The 10-byte product, V times rA, replaces registers A and X. The signs of rA 
and rX are both set to the algebraic sign of the product (namely, + if the signs 
of V and rA were the same, — if they were different). 


e DIV (divide). C = 4; F = field. 

The value of rA and rX, treated as a 10-byte number rAX with the sign of rA, 
is divided by the value V. If V = 0 or if the quotient is more than five bytes in 
magnitude (this is equivalent to the condition that |rA| > |V|), registers A and X 
are filled with undefined information and the overflow toggle is set on. Otherwise 
the quotient +||rAX/V|| is placed in rA and the remainder +(|rAX| mod |V|) 
is placed in rX. The sign of rA afterwards is the algebraic sign of the quotient 
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(namely, + if the signs of V and rA were the same, — if they were different). 
The sign of rX afterwards is the previous sign of rA. 


Examples of arithmetic instructions: In most cases, arithmetic is done only 
with MIX words that are single five-byte numbers, not packed with several fields. 
It is, however, possible to operate arithmetically on packed MIX words, if some 
caution is used. The following examples should be studied carefully. (As before, 
? designates an unknown value.) 


+|1234/1] 150| rA before 
+| 1005| 50] Cell 1000 
ADD 1000 + |1334]6| 200| rA after 


-]1234]0]0]9]| rA before 
-|2000 150]0] Cell 1000 


SUB 1000 +) 766| 149 |? rA after 
+/1/1/1]1]1] rA before 
+/41/41/1/1]1] Cell 1000 

MUL 1000 +/0/1/2/3)4 rA after 
+/5/4/3]/2/1] rX after 


z 112 | rA before 
?|2|?|?|?]|?| Cell 1000 
MUL 1000(1:1) = O| rA after 
- 224| rX after 


-150| 0| 1124| rA before 
-|2/0/0/0]0] Cell 1000 
MUL 1000 +| 100/0}] 224] rA after 
+/8/0/0]0]0]| rX after 


+ O| rA before 

? S 17 | rX before 

+ n 3| Cell 1000 
DIV 1000 + = 5| rA after 


+ 2| rX after 
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- O| rA before 
+ 1235 0|3]|1| rX before 
-|0}0/0/2]/0] Cell 1000 
DIV 1000 +10] 617|?|?| rA after 
-/0/0/0]?/1) rX after 


(These examples have been prepared with the philosophy that it is better to give 
a complete, baffling description than an incomplete, straightforward one.) 


Address transfer operators. In the following operations, the (possibly in- 
dexed) “address” M is used as a signed number, not as the address of a cell in 
memory. 

e ENTA (enter A). C = 48; F = 2. 

The quantity M is loaded into rA. The action is equivalent to ‘LDA’ from a 
memory word containing the signed value of M. If M = 0, the sign of the 
instruction is loaded. 

Examples: ‘ENTA 0’ sets rA to zeros, with a + sign. ‘ENTA 0,1’ sets rA 
to the current contents of index register 1, except that —0 is changed to +0. 
‘ENTA -0,1 is similar, except that +0 is changed to —0. 

e ENTX (enter X). C = 55; F = 2. 

e ENTi (enter i). C = 48 + i; F = 2. 

Analogous to ENTA, loading the appropriate register. 

e ENNA (enter negative A). C = 48; F = 3. 

e ENNX (enter negative X). C = 55; F = 3. 

e ENNi (enter negative i). C = 48 + i; F = 3. 

Same as ENTA, ENTX, and ENTi, except that the opposite sign is loaded. 

Example: ‘ENN3 0,3’ replaces rI3 by its negative, although —0 remains —0. 


e INCA (increase A). C = 48; F = 0. 
The quantity M is added to rA; the action is equivalent to ‘ADD’ from a memory 
word containing the value of M. Overflow is possible and it is treated just as 
in ADD. 

Example: ‘INCA 1’ increases the value of rA by one. 
e INCX (increase X). C = 55; F = 0. 
The quantity M is added to rX. If overflow occurs, the action is equivalent to 
ADD, except that rX is used instead of rA. Register A is never affected by this 
instruction. 
e INCi (increase i). C = 48 + i; F = 0. 
Add M to rli. Overflow must not occur; if M + rIi doesn’t fit in two bytes, the 
result of this instruction is undefined. 
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e DECA (decrease A). C = 48; F = 1. 
e DECX (decrease X). C = 55; F = 1. 
e DECi (decrease i). C = 48 + i; F = 1. 
These eight instructions are the same as INCA, INCX, and INCi, respectively, 
except that M is subtracted from the register rather than added. 

Notice that the operation code C is the same for ENTA, ENNA, INCA, and 
DECA; the F-field is used to distinguish the various operations from each other. 


Comparison operators. MIX’s comparison operators all compare the value 
contained in a register with a value contained in memory. The comparison 
indicator is then set to LESS, EQUAL, or GREATER according to whether the value 
of the register is less than, equal to, or greater than the value of the memory 
cell. A minus zero is equal to a plus zero. 

e CMPA (compare A). C = 56; F = field. 

The specified field of rA is compared with the same field of CONTENTS(M). If 
F does not include the sign position, the fields are both considered nonnegative; 
otherwise the sign is taken into account in the comparison. (An equal comparison 
always occurs when F is (0:0), since minus zero equals plus zero.) 

e CMPX (compare X). C = 63; F = field. 

This is analogous to CMPA. 


e CMPi (compare i). C = 56 + i; F = field. 
Analogous to CMPA. Bytes 1, 2, and 3 of the index register are treated as zero in 
the comparison. (Thus if F = (1:2), the result cannot be GREATER.) 


Jump operators. Instructions are ordinarily executed in sequential order; in 
other words, the command that is performed after the command in location P 
is usually the one found in location P + 1. But several “jump” instructions 
allow this sequence to be interrupted. When a typical jump takes place, the 
J-register is set to the address of the next instruction (that is, to the address of 
the instruction that would have been next if we hadn’t jumped). A “store J” 
instruction then can be used by the programmer, if desired, to set the address 
field of another command that will later be used to return to the original place 
in the program. The J-register is changed whenever a jump actually occurs in 
a program, except when the jump operator is JSJ, and it is never changed by 
non-jumps. 

e JMP (jump). C = 39; F = 0. 

Unconditional jump: The next instruction is taken from location M. 

e JSJ (jump, save J). C = 39; F = 1. 

Same as JMP except that the contents of rJ are unchanged. 

e JOV (jump on overflow). C = 39; F = 2. 

If the overflow toggle is on, it is turned off and a JMP occurs; otherwise nothing 
happens. 

e JNOV (jump on no overflow). C = 39; F = 3. 

If the overflow toggle is off, a JMP occurs; otherwise it is turned off. 
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e JL, JE, JG, JGE, JNE, JLE (jump on less, equal, greater, greater-or-equal, 
unequal, less-or-equal). C = 39; F = 4, 5, 6, 7, 8, 9, respectively. 

Jump if the comparison indicator is set to the condition indicated. For example, 
JNE will jump if the comparison indicator is LESS or GREATER. The comparison 
indicator is not changed by these instructions. 

e JAN, JAZ, JAP, JANN, JANZ, JANP (jump A negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 40; F = 0, 1, 2, 3, 4, 5, respectively. 

If the contents of rA satisfy the stated condition, a JMP occurs, otherwise nothing 
happens. “Positive” means greater than zero (not zero); “nonpositive” means 
the opposite, namely zero or negative. 

e JXN, JXZ, IXP, JXNN, JXNZ, JXNP (jump X negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 47; F = 0, 1, 2, 3, 4, 5, respectively. 

e JiN, JiZ, JiP, JiNN, JiNZ, JiNP (jump i negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 40 + i; F = 0, 1, 2, 3, 4, 5, respectively. These 42 
instructions are analogous to the corresponding operations for rA. 


Miscellaneous operators. 

e SLA, SRA, SLAX, SRAX, SLC, SRC (shift left A, shift right A, shift left AX, shift 
right AX, shift left AX circularly, shift right AX circularly). C = 6; F = 0, 1, 2, 
3, 4, 5, respectively. 

These six are the “shift” commands, in which M specifies a number of MIX bytes 
to be shifted left or right; M must be nonnegative. SLA and SRA do not affect rX; 
the other shifts affect both registers A and X as though they were a single 10- 
byte register. With SLA, SRA, SLAX, and SRAX, zeros are shifted into the register 
at one side, and bytes disappear at the other side. The instructions SLC and SRC 
call for a “circulating” shift, in which the bytes that leave one end enter in at 
the other end. Both rA and rX participate in a circulating shift. The signs of 
registers A and X are not affected in any way by any of the shift commands. 


Examples: Register A Register X 
Initial contents |+/1])2/3])4)5 -|6/7/8}9 10 
SRAX 1 +/O0/1/2/3)4 -|/5/6/7)8]9 
SLA 2 +/2)/3}4/0]0 -|/5/6/7)8]9 
SRC 4 +/6|/7/8/9}]2 -|3)/4/0])0]5 
SRA 2 +/0/0/6/7]8 -|3)/4/0)0]5 
SLC 501 +/0/6/7)/8]3 -|4/0/0]5]0 
e MOVE. C = 7; F = number, normally 1. 
The number of words specified by F is moved, starting from location M to the 


location specified by the contents of index register 1. The transfer occurs one 
word at a time, and rI1 is increased by the value of F at the end of the operation. 
If F = 0, nothing happens. 

Care must be taken when there’s overlap between the locations involved; for 
example, suppose that F = 3 and M = 1000. Then if rll = 999, we transfer 
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CONTENTS (1000) to CONTENTS (999), CONTENTS(1001) to CONTENTS (1000), and 
CONTENTS (1002) to CONTENTS (1001); nothing unusual occurred here. But if rI1 
were 1001 instead, we would move CONTENTS(1000) to CONTENTS (1001), then 
CONTENTS (1001) to CONTENTS (1002), then CONTENTS (1002) to CONTENTS (1003), 
so we would have moved the same word CONTENTS (1000) into three places. 


e NOP (no operation). C = 0. 

No operation occurs, and this instruction is bypassed. F and M are ignored. 

e HLT (halt). C = 5; F = 2. 

The machine stops. When the computer operator restarts it, the net effect is 
equivalent to NOP. 


Input-output operators. MIX has a fair amount of input-output equipment 
(all of which is optional at extra cost). Each device is given a number as follows: 


Unit number Peripheral device Block size 
t Tape unit number t (0 < t < 7) 100 words 
d Disk or drum unit number d (8 < d < 15) 100 words 
16 Card reader 16 words 
17 Card punch 16 words 
18 Line printer 24 words 
19 Typewriter terminal 14 words 
20 Paper tape 14 words 


Not every MIX installation will have all of this equipment available; we will 
occasionally make appropriate assumptions about the presence of certain devices. 
Some devices may not be used both for input and for output. The number of 
words mentioned in the table above is a fixed block size associated with each unit. 

Input or output with magnetic tape, disk, or drum units reads or writes full 
words (five bytes and a sign). Input or output with units 16 through 20, however, 
is always done in a character code where each byte represents one alphameric 
character. Thus, five characters per MIX word are transmitted. The character 
code is given at the top of Table 1, which appears at the close of this section and 
on the end papers of this book. The code 00 corresponds to ‘u’, which denotes 
a blank space. Codes 01-29 are for the letters A through Z with a few Greek 
letters thrown in; codes 30-39 represent the digits 0, 1, ..., 9; and further codes 
40, 41, ... represent punctuation marks and other special characters. (MIX’s 
character set harks back to the days before computers could cope with lowercase 
letters.) We cannot use character code to read in or write out all possible values 
that a byte may have, since certain combinations are undefined. Moreover, some 
input-output devices may be unable to handle all the symbols in the character 
set; for example, the symbols ° and “ that appear amid the letters will perhaps 
not be acceptable to the card reader. When character-code input is being done, 
the signs of all words are set to +; on output, signs are ignored. If a typewriter is 
used for input, the “carriage return” that is typed at the end of each line causes 
the remainder of that line to be filled with blanks. 
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The disk and drum units are external memory devices each containing 100- 
word blocks. On every IN, OUT, or IOC instruction as defined below, the particular 
100-word block referred to by the instruction is specified by the current contents 
of rX, which should not exceed the capacity of the disk or drum involved. 

e IN (input). C = 36; F = unit. 

This instruction initiates the transfer of information from the input unit specified 
into consecutive locations starting with M. The number of locations transferred 
is the block size for this unit (see the table above). The machine will wait at 
this point if a preceding operation for the same unit is not yet complete. The 
transfer of information that starts with this instruction will not be complete 
until an unknown future time, depending on the speed of the input device, so a 
program must not refer to the information in memory until then. It is improper 
to attempt to read any block from magnetic tape that follows the latest block 
written on that tape. 

e OUT (output). C = 37; F = unit. 

This instruction starts the transfer of information from memory locations start- 
ing at M to the output unit specified. The machine waits until the unit is ready, 
if it is not initially ready. The transfer will not be complete until an unknown 
future time, depending on the speed of the output device, so a program must 
not alter the information in memory until then. 

e IOC (input-output control). C = 35; F = unit. 

The machine waits, if necessary, until the specified unit is not busy. Then a 
control operation is performed, depending on the particular device being used. 
The following examples are used in various parts of this book: 

Magnetic tape: If M = 0, the tape is rewound. If M < 0 the tape is skipped 
backward —M blocks, or to the beginning of the tape, whichever comes first. 
If M > 0, the tape is skipped forward; it is improper to skip forward over any 
blocks following the one last written on that tape. 

For example, the sequence ‘OUT 1000(3); IOC -1(3); IN 2000(3)’ writes 
out one hundred words onto tape 3, then reads it back in again. Unless the 
tape reliability is questioned, the last two instructions of that sequence are only 
a slow way to move words 1000-1099 to locations 2000-2099. The sequence 
‘OUT 1000(3); IOC +1(3)’ is improper. 

Disk or drum: M should be zero. The effect is to position the device 
according to rX so that the next IN or OUT operation on this unit will take 
less time if it uses the same rX setting. 

Line printer: M should be zero. ‘IOC 0(18)’ skips the printer to the top of 
the following page. 

Paper tape: M should be zero. ‘IOC 0(20)’ rewinds the tape. 

e JRED (jump ready). C = 38; F = unit. 

A jump occurs if the specified unit is ready, that is, finished with the preceding 
operation initiated by IN, OUT, or IOC. 

e JBUS (jump busy). C = 34; F = unit. 

Analogous to JRED, but the jump occurs when the specified unit is not ready. 
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Example: In location 1000, the instruction ‘JBUS 1000(16)’ will be executed 
repeatedly until unit 16 is ready. 


The simple operations above complete MIX’s repertoire of input-output in- 
structions. There is no “tape check” indicator, etc., to cover exceptional con- 
ditions on the peripheral devices. Any such condition (e.g., paper jam, unit 
turned off, out of tape, etc.) causes the unit to remain busy, a bell rings, and 
the skilled computer operator fixes things manually using ordinary maintenance 
procedures. Some more complicated peripheral units, which are more expensive 
and more representative of contemporary equipment than the fixed-block-size 
tapes, drums, and disks described here, are discussed in Sections 5.4.6 and 5.4.9. 


Conversion Operators. 


e NUM (convert to numeric). C = 5; F = 0. 

This operation is used to change the character code into numeric code. M is 
ignored. Registers A and X are assumed to contain a 10-byte number in character 
code; the NUM instruction sets the magnitude of rA equal to the numerical value 
of this number (treated as a decimal number). The value of rX and the sign of 
rA are unchanged. Bytes 00, 10, 20, 30, 40, ... convert to the digit zero; bytes 
01, 11, 21, ... convert to the digit one; etc. Overflow is possible, and in this case 
the remainder modulo bř is retained, where b is the byte size. 


e CHAR (convert to characters). C = 5; F = 1. 

This operation is used to change numeric code into character code suitable for 
output to punched cards or tape or the line printer. The value in rA is converted 
into a 10-byte decimal number that is put into registers A and X in character 
code. The signs of rA and rX are unchanged. M is ignored. 


Examples: Register A Register X 
Initial contents - | 0O | 00 | 31 | 32 | 39 + | 37 | 57 | 47 | 30 | 30 
NUM 0O = 12977700 + | 37 | 57 | 47 | 30 | 30 
INCA 1 = | | 12977699 + | 37 | 57 | 47 | 30 | 30 
CHAR O - | 30 | 30 | 31 | 32 | 39 + | 37 | 37 | 36 | 39 | 39 


Timing. To give quantitative information about the efficiency of MIX programs, 
each of MIX’s operations is assigned an execution time typical of vintage-1970 
computers. 

ADD, SUB, all LOAD operations, all STORE operations (including STZ), all shift 
commands, and all comparison operations take two units of time. MOVE requires 
one unit plus two for each word moved. MUL, NUM, CHAR each require 10 units and 
DIV requires 12. The execution time for floating point operations is specified in 
Section 4.2.1. All remaining operations take one unit of time, plus the time the 
computer may be idle on the IN, OUT, IOC, or HLT instructions. 

Notice in particular that ENTA takes one unit of time, while LDA takes two 
units. The timing rules are easily remembered because of the fact that, except 
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for shifts, conversions, MUL, and DIV, the number of time units equals the number 
of references to memory (including the reference to the instruction itself). 

MIX’s basic unit of time is a relative measure that we will denote simply 
by u. It may be regarded as, say, 10 microseconds (for a relatively inexpensive 
computer) or as 10 nanoseconds (for a relatively high-priced machine). 


Example: The sequence LDA 1000; INCA 1; STA 1000 takes exactly 5u. 


And now I see with eye serene 
The very pulse of the machine. 


— WILLIAM WORDSWORTH, 
She Was a Phantom of Delight (1804) 


Summary. We have now discussed all the features of MIX, except for its 
“GO button,” which is discussed in exercise 26. Although MIX has nearly 150 
different operations, they fit into a few simple patterns so that they can easily 
be remembered. Table 1 summarizes the operations for each C-setting. The 
name of each operator is followed in parentheses by its default F-field. 

The following exercises give a quick review of the material in this section. 
They are mostly quite simple, and the reader should try to do nearly all of them. 


EXERCISES 
1. [00] If MIX were a ternary (base 3) computer, how many “trits” would there be 
per byte? 
2. [02] If a value to be represented within MIX may get as large as 99999999, how 
many adjacent bytes should be used to contain this quantity? 


3. [02] Give the partial field specifications, (L:R), for the (a) address field, (b) index 
field, (c) field field, and (d) operation code field of a MIX instruction. 


4. [00] The last example in (5) is ‘LDA -2000,4’. How can this be legitimate, in 
view of the fact that memory addresses should not be negative? 


5. [10] What symbolic notation, analogous to (4), corresponds to (6) if (6) is re- 
garded as a MIX instruction? 


6. [10] Assume that location 3000 contains 


+]5]1] 200 |15] . 


What is the result of the following instructions? (State if any of them are undefined 
or only partially defined.) (a) LDAN 3000; (b) LD2N 3000(3:4); (c) LDX 3000(1:3); 
(d) LD6 3000; (e) LDXN 3000(0:0). 

7. [M15] Give a precise definition of the results of the DIV instruction for all cases 
in which overflow does not occur, using the algebraic operations X mod Y and |X/Y]. 


8. [15] The last example of the DIV instruction that appears on page 133 has “rX 


before” equal to | + 1235|0/3|1). If this were | -]1234]0|3]1 instead, but 


other parts of that example were unchanged, what would registers A and X contain 
after the DIV instruction? 
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Table 1 
. |00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 
Character code: | A BC D-E-F’G@ HT? S°K LM ONO POO R STU 
00 1 01 2 02 2 03 10 
No operation rA+ rA+ V rA+ rA—-V rAX + rAx V 
ADD (0:5) SUB(0:5) MUL (0:5) 
NOP (0) FADD (6) FSUB(6) FMUL (6) 

08 2 09 2 10 2 11 2 
rA V ril + V rIi2 + V r13 + V 
LDA(O:5) LD1(0:5) LD2(0:5) LD3 (0:5) 

16 2 17 2 18 2 19 2 
rA¢ -V rll + —V rI2 + —V r13 + —V 
LDAN (0:5) LD1N(0:5) LD2N (0:5) LD3N (0:5) 

24 2 25 2 26 2 27 2 

M(F) = rA M(F) + rll M(F) + r12 M(F) © rI3 
STA (0:5) ST1(0:5) ST2(0:5) ST3 (0:5) 

32 2 33 2 34 1 35 1+T 
M(F) « rJ M(F) + 0 Unit F busy? Control, unit F 
STJ(0:2) STZ(0:5) JBUS (0) I0C(0) 

40 1 41 1 42 1 43 1 

rA:0, jump rI1:0, jump rI2:0, jump rI3:0, jump 

JA[+] Ji] J2[+] J3 [+] 
48 1 49 1 50 1 51 1 
rA & [rA]?+M ril © [rl]? +M r12 + [r12]? +M rI3 & [r13]? + M 


INCA(O) DECA(1) 
ENTA(2) ENNA(3) 


INC1(0) DEC1(1) 
ENT1(2) ENN1(3) 


INC2(0) DEC2(1) 
ENT2(2) ENN2(3) 


INC3(0) DEC3(1) 
ENT3(2) ENN3(3) 


56 2 57 2 58 2 59 2 
CI + rA(F):V CI & rll(F):V CI + rI2(F): V CI + rI3(F): V 
CMPA (0:5) : : i 
FCMP (6) CMP1 (0:5) CMP2 (0:5) CMP3 (0:5) 


General form: 


| C t 


Description 


OP (F) 


C = operation code, (5:5) field of instruction 

F = op variant, (4:4) field of instruction 

M = address of instruction after indexing 

V = M(F) = contents of F field of location M 

OP = symbolic name for operation 
(F) = normal F setting 


t = execution time; T = interlock time 
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25 26 27 28 29 30 31 32 33 34 35 


36 37 38 39 40 41 42 43 44 45 


46 47 48 49 50 51 52 53 54 55 


VwxyYyzoi23456789 » €)+- * / $< > @ 5: , 
04 12 05 10 06 2 07 1+2F 
rA + rAX/V Special Shift M bytes Move F words 

rX + remainder NUM(O) SLA(0) SRA(1) from M to rI1 

DIV(0:5) CHAR (1) SLAX(2) SRAX(3) MOVE(1) 
FDIV(6) HLT (2) SLC(4) SRC(5) 

12 2 13 2 14 2 15 2 
ri4 + V rI5 + V rl6¢+ V rX V 
LD4(0:5) LD5(0:5) LD6 (0:5) LDX (0:5) 

20 2 21 2 22 2 23 2 
ri4 + -V rl5 + -V rl6 + -V rX + -V 
LD4N (0:5) LD5N (0:5) LD6N (0:5) LDXN (0:5) 

28 2 29 2 30 2 31 2 

M(F) ¢ 114 M(F) & rI5 M(F) + rI6 M(F) rX 
ST4 (0:5) ST5(0:5) ST6 (0:5) STX (0:5) 

36 1+T 37 1+T 38 1 39 1 

Input, unit F Output, unit F Unit F ready? Jumps 


JMP(0) JSJ(1) 


IN(0) OUT (0) JRED (0) JOV(2) JNOV(3) 
also [*] below 
44 1 45 1 46 1 47 1 
rl4:0, jump rI5:0, jump rl6:0, jump rX:0, jump 
J4[+] J5[+] J6 [+] JX [+] 
52 1 53 1 54 ft 55 1 


rl4 + [r14]? +M 


INC4(0) DEC4(1) 
ENT4(2) ENN4(3) 


rI5 + [r15]? +M 


INC5(0) DEC5(1) 
ENT5(2) ENN5(3) 


rl6 < [r16]? + M 


INC6(0) DEC6(1) 
ENT6 (2) ENN6 (3) 


rX + [X]? +M 


INCX(0) DECX(1) 
ENTX(2) ENNX(3) 


60 2 61 2 62 2 63 2 
CI + rl4(F):V CI + rI5(F): V CI + rl6(F):V CI + rX(F):V 
CMP4 (0:5) CMP5 (0:5) CMP6 (0:5) CMPX (0:5) 
[*]: [+]: 
rA = register A JL(4) < N(0) 
rX = register X JE(5) = Z(1) 
rAX = registers A and X as one JG(6) > P(2) 
rli = index register i, 1 < i < 6 JGE(7) > NN(3) 
rJ = register J JNE(8) # NZ(4) 
CI = comparison indicator JLE(9) < NP(5) 
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> 9. [15] List all the MIX operators that can possibly affect the setting of the overflow 
toggle. (Do not include floating point operators.) 


10. [15] List all the MIX operators that can possibly affect the setting of the compar- 
ison indicator. 
> 11. [15] List all the MIX operators that can possibly affect the setting of rI1. 


12. [10] Find a single instruction that has the effect of multiplying the current con- 
tents of r13 by two and leaving the result in rI3. 


> 13. [10] Suppose location 1000 contains the instruction ‘JOV 1001’. This instruction 
turns off the overflow toggle if it is on (and the next instruction executed will be in 
location 1001, in any case). If this instruction were changed to ‘JNOV 1001’, would 
there be any difference? What if it were changed to ‘JOV 1000’ or ‘JNOV 1000’? 


14. [20] For each MIX operation, consider whether there is a way to set the +AA, 
I, and F portions so that the result of the instruction is precisely equivalent to NOP 
(except that the execution time may be longer). Assume that nothing is known about 
the contents of any registers or any memory locations. Whenever it is possible to 
produce a NOP, state how it can be done. Examples: INCA is a no-op if the address and 
index parts are zero. JMP can never be a no-op, since it affects rJ. 


15. [10] How many alphameric characters are there in a typewriter or paper-tape 
block? in a card-reader or card-punch block? in a line-printer block? 


16. [20] Write a program that sets memory cells 0000-0099 all to zero and is (a) as 
short a program as possible; (b) as fast a program as possible. [Hint: Consider using 
the MOVE command.| 

17. [26] This is the same as the previous exercise, except that locations 0000 through 
N, inclusive, are to be set to zero, where N is the current contents of rl2. Your 
programs (a) and (b) should work for any value 0 < N < 2999; they should start in 
location 3000. 


> 18. [22] After the following “number one” program has been executed, what changes 
to registers, toggles, and memory have taken place? (For example, what is the final 
setting of r11? of rX? of the overflow and comparison indicators?) 
STZ 1 
ENNX 1 
STX 1(0:1) 
SLAX 
ENNA 1 
INCX 1 
ENT1 1 
1 
1 


m. 


SRC 
ADD 

DEC1 -1 

STZ 1 

CMPA 1 

MOVE -1,1(1) 
NUM 1 

CHAR 1 

HLT 1 J 


> 19. [14] What is the execution time of the program in the preceding exercise, not 
counting the HLT instruction? 
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20. [20] Write a program that sets all 4000 memory cells equal to a ‘HLT’ instruction, 
and then stops. 


21. [24] (a) Can the J-register ever be zero? (b) Write a program that, given a number 
N in rl4, sets register J equal to N, assuming that 0 < N < 3000. Your program should 
start in location 3000. When your program has finished its execution, the contents of 
all memory cells must be unchanged. 

22. [28] Location 2000 contains an integer number, X. Write two programs that 
compute X'? and halt with the result in register A. One program should use the 
minimum number of MIX memory locations; the other should require the minimum 
execution time possible. Assume that X"? fits into a single word. 


23. [27] Location 0200 contains a word 


+|aļb|cļ|dļelļ|; 


write two programs that compute the “reflected” word 


+leļdic|bļa 


and halt with the result in register A. One program should do this without using 
MIX’s ability to load and store partial fields of words. Both programs should take the 
minimum possible number of memory locations under the stated conditions (including 
all locations used for the program and for temporary storage of intermediate results). 


24. [21] Assuming that registers A and X contain 


+/O;/a|b|cld and t+le;/fig]hlz|, 


respectively, write two programs that change the contents of these registers to 


+|aļb|cidjļe and +loifiglhlil, 


respectively, using (a) minimum memory space and (b) minimum execution time. 


25. [30] Suppose that the manufacturer of MIX wishes to come out with a more 
powerful computer (“Mixmaster”?), and he wants to convince as many as possible 
of those people now owning a MIX computer to invest in the more expensive machine. 
He wants to design this new hardware to be an eztension of MIX, in the sense that 
all programs correctly written for MIX will work on the new machines without change. 
Suggest desirable things that could be incorporated in this extension. (For example, 
can you make better use of the I-field of an instruction?) 


26. [32] This problem is to write a card-loading routine. Every computer has its own 
peculiar “bootstrapping” problems for getting information initially into the machine 
and for starting a job correctly. In MIX’s case, the contents of a card can be read only 
in character code, and the cards that contain the loading program itself must meet this 
restriction. Not all possible byte values can be read from a card, and each word read 
in from cards is positive. 

MIX has one feature that has not been explained in the text: There is a “GO 
button,” which is used to get the computer started from scratch when its memory 
contains arbitrary information. When this button is pushed by the computer operator, 
the following actions take place: 


1) A single card is read into locations 0000-0015; this is essentially equivalent to the 
instruction ‘IN 0(16)’. 
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2) When the card has been completely read and the card reader is no longer busy, 
a JMP to location 0000 occurs. The J-register is also set to zero, and the overflow 
toggle is cleared. 

3) The machine now begins to execute the program it has read from the card. 


Note: MIX computers without card readers have their GO-button attached to another 
input device. But in this problem we will assume the presence of a card reader, unit 16. 
The loading routine to be written must satisfy the following conditions: 


i) The input deck should begin with the loading routine, followed by information 
cards containing the numbers to be loaded, followed by a “transfer card” that shuts 
down the loading routine and jumps to the beginning of the program. The loading 
routine should fit onto two cards. 

ii) The information cards have the following format: 


Columns 1-5, ignored by the loading routine. 

Column 6, the number of consecutive words to be loaded on this card (a num- 
ber between 1 and 7, inclusive). 

Columns 7-10, the location of word 1, which is always greater than 100 (so 
that it does not overlay the loading routine). 

Columns 11-20, word 1. 

Columns 21-30, word 2 (if column 6 > 2). 


Columns 71-80, word 7 (if column 6 = 7). 
The contents of words 1, 2, ... are punched numerically as decimal numbers. If a word 
is to be negative, a minus (“11-punch”) is overpunched over the least significant digit, 
e.g., in column 20. Assume that this causes the character code input to be 10, 11, 12, 
..., 19 rather than 30, 31, 32, ..., 39. For example, a card that has 


ABCDE31000012345678900000000010000000100 
punched in columns 1-40 should cause the following data to be loaded: 
1000: +0123456789; 1001: +0000000001; 1002: —0000000100. 


iii) The transfer card has the format TRANSOnnnn in columns 1-10, where nnnn is the 
place where execution should start. 

iv) The loading routine should work for all byte sizes without any changes to the cards 
bearing the loading routine. No card should contain any of the characters corresponding 
to bytes 20, 21, 48, 49, 50, ... (namely, the characters °, “, =, $, <, ...), since these 
characters cannot be read by all card readers. In particular, the ENT, INC, and CMP 
instructions cannot be used; they can’t necessarily be punched on a card. 


1.3.2. The MIX Assembly Language 


A symbolic language is used to make MIX programs considerably easier to read 
and to write, and to save the programmer from worrying about tedious cleri- 
cal details that often lead to unnecessary errors. This language, MIXAL (“MIX 
Assembly Language”), is an extension of the notation used for instructions in 
the previous section. Its main features are the optional use of alphabetic names 
to stand for numbers, and a location field to associate names with memory 
locations. 
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MIXAL can readily be comprehended if we consider first a simple example. 
The following code is part of a larger program; it is a subroutine to find the 
maximum of n elements X[1],...,X[n], according to Algorithm 1.2.10M. 


Program M (Find the mazimum). Register assignments: rA = m, rll = n, 
rl2 = j, rI3 =k, X[i] = CONTENTS(X + i). 


Assembled instructions Line no. LOC OP ADDRESS ‘Times Remarks 
01 X EQU 1000 
02 ORIG 3000 
3000: | + 3009 0|2/32 03 MAXIMUM STJ EXIT 1 Subroutine linkage 
3001: | + l 0;1;2)51 04 INIT ENT3 0,1 1 M1. Initialize. k + n. 
3002: | + 3005 0/0}39} 05 JMP CHANGEM 1 jen, m+ Xjn], ken-1. 
3003: | + 1000 3/5 |56 06 LOOP CMPA X,3 n—1 M3. Compare. 
3004: |+]|3007|0[7]39] 07 JGE *+3 n—1 ToM5ifm > X[k]. 
3005: | + l 0/3/2]|50 08 CHANGEM ENT2 0,3 A+1 M4. Change m. j <4 k. 
3006: |+|1000|3[5]o8| 09 LDA X,3 A+1 m+ X[k]. 
3007: | + l 1/0|/1/51 10 DEC3 1 n M5. Decrease k. 
3008: |+ 3003 0/2/43 11 J3P LOOP n M2. All tested? To M3 if k>0. 
3009: | + 3009 0/0] 39 12 EXIT JMP * 1 Return to main program. |] 


This program is an example of several things simultaneously: 


a) The columns headed “LOC”, “OP”, and “ADDRESS” are of principal interest; 
they contain a program in the MIXAL symbolic machine language, and we shall 
explain the details of this program below. 


b) The column headed “Assembled instructions” shows the actual numeric 
machine language that corresponds to the MIXAL program. MIXAL has been 
designed so that any MIXAL program can easily be translated into numeric 
machine language; the translation is usually carried out by another computer 
program called an assembly program or assembler. Thus, programmers may 
do all of their machine language programming in MIXAL, never bothering to 
determine the equivalent numeric codes by hand. Virtually all MIX programs in 
this book are written in MIXAL. 


c) The column headed “Line no.” is not an essential part of the MIXAL program; 
it is merely included with MIXAL examples in this book so that we can readily 
refer to parts of the program. 


d) The column headed “Remarks” gives explanatory information about the 
program, and it is cross-referenced to the steps of Algorithm 1.2.10M. The reader 
should compare that algorithm (page 96) with the program above. Notice that 
a little “programmer’s license” was used during the transcription into MIX code; 
for example, step M2 has been put last. The “register assignments” stated at 
the beginning of Program M show what components of MIX correspond to the 
variables in the algorithm. 


e) The column headed “Times” will be instructive in many of the MIX programs 
we will be studying in this book; it represents the profile, the number of times the 
instruction on that line will be executed during the course of the program. Thus, 
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line 06 will be performed n—1 times, etc. From this information we can determine 
the length of time required to perform the subroutine; it is (5+5n+3A)u, where 
A is the quantity that was carefully analyzed in Section 1.2.10. 

Now let’s discuss the MIXAL part of Program M. Line 01, 


X EQU 1000, 


says that symbol X is to be equivalent to the number 1000. The effect of this may 
be seen on line 06, where the numeric equivalent of the instruction ‘CMPA X,3’ 
appears as 


+|1000] 3] 5 |56| , 


that is, ‘CMPA 1000,3’. 

Line 02 says that the locations for succeeding lines should be chosen sequen- 
tially, originating with 3000. Therefore the symbol MAXIMUM that appears in the 
LOC field of line 03 becomes equivalent to the number 3000, INIT is equivalent 
to 3001, LOOP is equivalent to 3003, etc. 

On lines 03 through 12 the OP field contains the symbolic names of MIX 
instructions: STJ, ENT3, etc. But the symbolic names EQU and ORIG, which 
appear in the OP column of lines 01 and 02, are somewhat different; EQU and 
ORIG are called pseudo-operations, because they are operators of MIXAL but not of 
MIX. Pseudo-operations provide special information about a symbolic program, 
without being instructions of the program itself. Thus the line 


X EQU 1000 


only talks about Program M, it does not signify that any variable is to be set 
equal to 1000 when the program is run. Notice that no instructions are assembled 
for lines 01 and 02. 

Line 03 is a “store J” instruction that stores the contents of register J into 
the (0:2) field of location EXIT. In other words, it stores rJ into the address part 
of the instruction found on line 12. 

As mentioned earlier, Program M is intended to be part of a larger program; 
elsewhere the sequence 


ENT1 100 
JMP MAXIMUM 
STA MAX 


would, for example, jump to Program M with n set to 100. Program M would 
then find the largest of the elements X[1],..., [100] and would return to the 
instruction ‘STA MAX’ with the maximum value in rA and with its position, j, in 
rI2. (See exercise 3.) 

Line 05 jumps the control to line 08. Lines 04, 05, 06 need no further 
explanation. Line 07 introduces a new notation: An asterisk (read “self”) refers 
to the location of the line on which it appears; ‘*+3’ (“self plus three”) therefore 
refers to three locations past the current line. Since line 07 is an instruction that 
corresponds to location 3004, the ‘*+3’ appearing there refers to location 3007. 

The rest of the symbolic code is self-explanatory. Notice the appearance of 
an asterisk again on line 12 (see exercise 2). 
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Our next example introduces a few more features of the assembly language. 
The object is to compute and print a table of the first 500 prime numbers, with 
10 columns of 50 numbers each. The table should appear as follows on the 
line printer: 


FIRST FIVE HUNDRED PRIMES 
0002 0233 0547 0877 1229 1597 1993 2371 2749 3187 
0003 0239 0557 0881 1231 1601 1997 2377 2753 3191 
0005 0241 0563 0883 1237 1607 1999 2381 2767 3203 
0007 0251 0569 0887 1249 1609 2003 2383 2777 3209 
0011 0257 0571 0907 1259 1613 2011 2389 2789 3217 


0229 0541 0863 1223 1583 1987 2357 2741 3181 3571 


We will use the following method. 


Algorithm P (Print table of 500 primes). This algorithm has two distinct 
parts: Steps P1-P8 prepare an internal table of 500 primes, and steps P9-P11 
print the answer in the form shown above. The latter part uses two “buffers,” 


in which line images are formed; while one buffer is being printed, the other one 
is being filled. 


P1. 


P2: 
P3. 
P4. 
P5. 


P6. 


P7: 


P8. 


P9. 


P10. 


P11. 


Start table.] Set PRIME[1] + 2, N + 3, J + 1. (In the following steps, 
N will run through the odd numbers that are candidates for primes; J will 
keep track of how many primes have been found so far.) 

N is prime.] Set J + J+ 1, PRIME[J] + N. 

500 found?] If J = 500, go to step P9. 

Advance N.] Set N + N + 2. 

K ¢ 2.] Set K + 2. (PRIME[K] will run through the possible prime divisors 
of N.) 

PRIME[K]\N?] Divide N by PRIME[K]; let Q be the quotient and R the 
remainder. If R = 0 (hence N is not prime), go to P4. 

PRIME[K] large?] If Q < PRIME[K], go to P2. (In such a case, N must 
be prime; the proof of this fact is interesting and a little unusual— see 
exercise 6.) 

Advance K.] Increase K by 1, and go to P6. 

Print title.) Now we are ready to print the table. Advance the printer 


to the next page. Set BUFFER[0] to the title line and print this line. Set 
Be 1Mel. 


Set up line.) Put PRIME[M], PRIME[50 +M], ..., PRIME[450+M] into 
BUFFER[B] in the proper format. 


Print line.] Print BUFFER[B]; set B «+ 1 — B (thereby switching to the 
other buffer); and increase M by 1. If M < 50, return to P10; otherwise the 
algorithm terminates. | 
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P1. Start table 


Y No 
P2. N is prime Yes (P7, PRIME [K] large? No (P6. PRIME [K] \N? 
Yes 
P3. 500 found? No P4. Advance N > P5. Ke2 
Yes 
Vv vA D 
P9. Print title > P10. Set up line > P11. Print line 
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P8. Advance K 


Fig. 14. Algorithm P. 


oe 


Program P (Print table of 500 primes). This program has deliberately been 
written in a slightly clumsy fashion in order to illustrate most of the features of 
MIXAL in a single program. rll = J — 500; rl2 = N; rI3 = K; rl4 indicates B; rI5 
is M plus multiples of 50. 


01 
02 
03 
04 
05 
06 
07 
08 
09 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 


* 
L 
PRINTER 
PRIME 
BUFO 
BUF 1 


START 


2H 


4H 


6H 


EQU 
EQU 
EQU 
EQU 
EQU 
ORI 
10C 
LD1 
LD2 


2 
Q 
H 


+] 
N 


— 
= 


2 
= 


>x H 
N 


YGQoQUHQUY g FH UM H 


G 


* EXAMPLE PROGRAM 


2000 
BUF0+25 
3000 

0 (PRINTER) 
=1{-L= 

=3= 

1 
PRIME+L, 1 
2F 

2 

2 

0 

0,2 
PRIME, 3 
4B 
PRIME, 3 

1 

6B 

2B 


TABLE OF PRIMES 


The number of primes to find 
Unit number of the line printer 
Memory area for table of primes 


Memory area for BUFFER[0] 
Memory area for BUFFER[1] 


Skip to new page. 

P1. Start table. J + 1. 
N¢ 3. 

P2. N is prime. J+ J+1. 
PRIME[J] + N. 

P3. 500 found? 

P4. Advance N. 

P5. K+ 2. 

P6. PRIME[K]\N? 
rAX +N. 
rA +Q, rX < R. 
To P4 if R= 0. 

P7. PRIME[K] large? 

P8. Advance K. 
To P6 if Q > PRIME[K]. 
Otherwise N is prime. 
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25 2H OUT TITLEC(PRINTER) P9. Print title. 

26 ENT4 BUF1+10 Set B & 1. 

a7 ENT5 -50 Set M+ 0. 

28 2H INC5 L+1 Advance M. 

29 4H LDA PRIME,5 P10. Set up line. (Right to left) 
30 CHAR Convert PRIME[M] to decimal. 
31 STX 0,4(1:4) 

32 DEC4 1 

33 DEC5 50 (rI5 goes down by 50 until 
34 J5P 4B it becomes nonpositive) 
35 OUT 0,4(PRINTER) P11. Print line. 

36 LD4 24,4 Switch buffers. 

37 J5N 2B If r15 = 0, we are done. 

38 HLT 

89 »* INITIAL CONTENTS OF TABLES AND BUFFERS 

40 ORIG PRIME+1 

41 CON 2 The first prime is 2. 

42 ORIG BUFO-5 

43 TITLE ALF FIRST Alphabetic information for 

44 ALF FIVE title line 

45 ALF HUND 

46 ALF RED P 

47 ALF RIMES 

48 ORIG BUFO+24 

49 CON BUF1+10 Each buffer refers to the other. 
50 ORIG BUF1+24 

51 CON BUFO+10 

52 END START End of routine. J 


The following points of interest should be noted about this program: 


1. Lines 01, 02, and 39 begin with an asterisk: This signifies a “comment” 
line that is merely explanatory, having no actual effect on the assembled program. 


2. As in Program M, the pseudo-operation EQU in line 03 sets the equivalent 
of a symbol; in this case, the equivalent of L is set to 500. (In the program of lines 
10-24, L represents the number of primes to be computed.) Notice that in line 
05 the symbol PRIME gets a negative equivalent; the equivalent of a symbol may 
be any signed five-byte number. In line 07 the equivalent of BUF1 is calculated 
as BUFO+25, namely 2025. MIXAL provides a limited amount of arithmetic on 
numbers; another example appears on line 13, where the value of PRIME+L (in 
this case, 499) is calculated by the assembly program. 


3. The symbol PRINTER has been used in the F-part on lines 09, 25, and 35. 
The F-part, which is always enclosed in parentheses, may be numeric or symbolic, 
just as the other portions of the ADDRESS field are. Line 31 illustrates the partial 
field specification ‘(1:4)’, using a colon. 


4. MIXAL provides several ways to specify non-instruction words. Line 41 
uses the pseudo-operation CON to specify an ordinary constant, ‘2’; the result of 
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line 41 is to assemble the word 


+ 2 


Line 49 shows a slightly more complicated constant, ‘BUF1+10’, which assembles 
as the word 


+] ° | "2035 


A constant may be enclosed in equal signs, in which case we call it a literal 
constant (see lines 10 and 11). The assembler automatically creates internal 
names and inserts ‘CON’ lines for literal constants. For example, lines 10 and 11 
of Program P are effectively changed to 


10 LD1 conl 
11 LD2 con2 


and then at the end of the program, between lines 51 and 52, the lines 


5la coni CON 1-L 
51b con2 CON 3 


are effectively inserted as part of the assembly procedure (possibly with con2 
first). Line 51a will assemble into the word 


-| © 499 


The use of literal constants is a decided convenience, because it means that 
programmers do not have to invent symbolic names for trivial constants, nor 
do they have to remember to insert constants at the end of each program. 
Programmers can keep their minds on the central problems and not worry 
about such routine details. (However, the literal constants in Program P aren’t 
especially good examples, because we would have had a slightly better program 
if we had replaced lines 10 and 11 by the more efficient commands ‘ENT1 1-L’ 
and ‘ENT2 3’.) 


5. A good assembly language should mimic the way a programmer thinks 
about machine programs. One example of this philosophy is the use of literal 
constants, as we have just mentioned; another example is the use of ‘*’, which 
was explained in Program M. A third example is the idea of local symbols such 
as the symbol 2H, which appears in the location field of lines 12, 25, and 28. 

Local symbols are special symbols whose equivalents can be redefined as 
many times as desired. A global symbol like PRIME has but one significance 
throughout a program, and if it were to appear in the location field of more than 
one line an error would be indicated by the assembler. But local symbols have a 
different nature; we write, for example, 2H (“2 here”) in the location field, and 
2F (“2 forward”) or 2B (“2 backward”) in the address field of a MIXAL line: 


2B means the closest previous location 2H; 
2F means the closest following location 2H. 
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Thus the ‘2F’ in line 14 refers to line 25; the ‘2B’ in line 24 refers back to line 
12; and the ‘2B’ in line 37 refers to line 28. An address of 2F or 2B never refers 
to its own line; for example, the three lines of MIXAL code 


2H EQU 10 
2H MOVE 2F(2B) 
2H EQU  2B-3 


are virtually equivalent to the single line 
MOVE *-3(10). 


The symbols 2F and 2B should never be used in the location field; the symbol 
2H should never be used in the address field. There are ten local symbols, which 
can be obtained by replacing ‘2’ in these examples by any digit from 0 to 9. 

The idea of local symbols was introduced by M. E. Conway in 1958, in 
connection with an assembly program for the UNIVAC I. Local symbols relieve 
programmers from the necessity of choosing symbolic names for every address, 
when all they want to do is refer to an instruction a few lines away. There often 
is no appropriate name for nearby locations, so programmers have tended to 
introduce meaningless symbols like X1, X2, X3, etc., with the potential danger 
of duplication. Local symbols are therefore quite useful and natural in an 
assembly language. 


6. The address part of lines 30 and 38 is blank. This means that the 
assembled address will be zero. We could have left the address blank in line 17 
as well, but the program would have been less readable without the redundant 0. 


7. Lines 43-47 use the pseudo-operation ALF, which creates a five-byte 
constant in MIX alphameric character code. For example, line 45 causes the 
word 


+ | 00 | 08 | 24 | 15 | 04 


to be assembled, representing ‘ HUND’ — part of the title line in Program P’s 
output. 

All locations whose contents are not specified in the MIXAL program are 
ordinarily set to positive zero (except the locations that are used by the loading 
routine, usually 3700-3999). Thus there is no need to set the other words of the 
title line to blanks, after line 47. 


8. Arithmetic may be used together with ORIG: See lines 40, 42, 48, and 50. 


9. The last line of a complete MIXAL program always has the OP-code ‘END’. 
The address on this line is the location at which the program is to begin, once 
it has been loaded into memory. 


10. As a final note about Program P, we can observe that the instructions 
have been organized so that index registers are counted towards zero, and tested 
against zero, whenever possible. For example, the quantity J-500, not J, is kept 
in rll. Lines 26-34 are particularly noteworthy, although perhaps a bit tricky. 
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* EXAMPLE PROGRAM ... TABLE OF PRIMES 
* 
L EQU 500 
PRINTER EQU 18 
PRIME EQU -1 
BUFO EQU 2000 
BUF1 EQU BUFO+25 

ORIG 3000 

START IOC 0(PRINTER) 

LD1 =1-L= 


Fig. 15. The first lines of Program P punched onto cards, or typed on a terminal. 


It may be of interest to note a few of the statistics observed when Program P 
was actually run. The division instruction in line 19 was executed 9538 times; 
the time to perform lines 10-24 was 182144u. 


MIXAL programs can be punched onto cards or typed on a computer terminal, 
as shown in Fig. 15. The following format is used in the case of punched cards: 


Columns 1-10 LOC (location) field; 

Columns 12-15 OP field; 

Columns 17-80 ADDRESS field and optional remarks; 
Columns 11, 16 blank. 


However, if column 1 contains an asterisk, the entire card is treated as a com- 
ment. The ADDRESS field ends with the first blank column following column 16; 
any explanatory information may be punched to the right of this first blank 
column with no effect on the assembled program. (Exception: When the OP field 
is ALF, the remarks always start in column 22.) 

When the input comes from a terminal, a less restrictive format is used: 
The LOC field ends with the first blank space, while the OP and ADDRESS fields 
(if present) begin with a nonblank character and continue to the next blank; the 
special OP-code ALF is, however, followed either by two blank spaces and five 
characters of alphameric data, or by a single blank space and five alphameric 
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characters, the first of which is nonblank. The remainder of each line contains 
optional remarks. 

The MIX assembly program accepts input files prepared in this manner and 
converts them to machine language programs in loadable form. Under favorable 
circumstances the reader will have access to a MIX assembler and MIX simulator, 
on which various exercises in this book can be worked out. 

Now we have seen what can be done in MIXAL. We conclude this section by 
describing the rules more carefully, and in particular we shall observe what is not 
allowed in MIXAL. The following comparatively few rules define the language. 


1. A symbol is a string of one to ten letters and/or digits, containing at least 
one letter. Examples: PRIME, TEMP, 20BY20. The special symbols dH, dF, and 
dB, where d is a single digit, will for the purposes of this definition be replaced 
by other unique symbols according to the “local symbol” convention described 
earlier. 


2. A number is a string of one to ten digits. Example: 00052. 


3. Each appearance of a symbol in a MIXAL program is said to be either a 
“defined symbol” or a “future reference.” A defined symbol is a symbol that has 
appeared in the LOC field of a preceding line of this MIXAL program. A future 
reference is a symbol that has not yet been defined in this way. 

4. An atomic expression is either 

a) a number, or 


b) a defined symbol (denoting the numerical equivalent of that symbol, see 
rule 13), or 


c) an asterisk (denoting the value of ®; see rules 10 and 11). 
5. An expression is either 

a) an atomic expression, or 

b) a plus or minus sign followed by an atomic expression, or 


c) an expression followed by a binary operation followed by an atomic expres- 


sion. 
The six admissible binary operations are +, -, *, /, //, and :. They are 
defined on numeric MIX words as follows: 
C = A+B LDA AA; ADD BB; STA CC. 
C = A-B LDA AA; SUB BB; STA CC. 
C = A*B LDA AA; MUL BB; STX CC. 
C = A/B LDA AA; SRAX 5; DIV BB; STA CC. 
C = A//B LDA AA; ENTX 0; DIV BB; STA CC. 
C = A:B LDA AA; MUL =8=; SLAX 5; ADD BB; STA CC. 


Here AA, BB, and CC denote locations containing the respective values of the 
symbols A, B, and C. Operations within an expression are carried out from left 
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to right. Examples: 


-1+5 equals 4. 
-1+5*20/6 equals 4*20/6 equals 80/6 equals 13 (going from left to right). 
1//3 equals a MIX word whose value is approximately b°/3 where 


b is the byte size; that is, a word representing the fraction 5 


with an assumed radix point at the left. 


1:3 equals 11 (usually used in partial field specification). 
*-3 equals ® minus three. 
kkk equals ® times ®. 


6. An A-part (which is used to describe the address field of a MIX instruction) 
is either 


a) vacuous (denoting the value zero), or 
b) an expression, or 


c) a future reference (denoting the eventual equivalent of the symbol; see 
rule 13), or 


d) a literal constant (denoting a reference to an internally created symbol; see 
rule 12). 
7. An index part (which is used to describe the index field of a MIX instruc- 
tion) is either 
a) vacuous (denoting the value zero), or 


b) a comma followed by an expression (denoting the value of that expression). 


8. An F-part (which is used to describe the F-field of a MIX instruction) is 
either 
a) vacuous (denoting the normal F-setting, based on the OP field as shown in 
Table 1.3.1-1), or 


b) a left parenthesis followed by an expression followed by a right parenthesis 
(denoting the value of the expression). 


9. A W-value (which is used to describe a full-word MIX constant) is either 
a) an expression followed by an F-part (in which case a vacuous F-part denotes 
(0:5)), or 
b) a W-value followed by a comma followed by a W-value of the form (a). 


A W-value denotes the value of a numeric MIX word determined as follows: 
Let the W-value have the form “E: (F1) ,Eo(Fo), ... ,En(Fn)”, where n > 1, 
the E’s are expressions, and the F’s are fields. The desired result is the final 
value that would appear in memory location WVAL if the following hypothetical 
program were executed: 


STZ WVAL; LDA C1; STA WVAL(F)); ...; LDA Cn; STA WVAL(F,,). 
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Here Cy, ..., Cn denote locations containing the values of expressions Ej, ..., 
En. Each F; must have the form 8L; + R; where 0 < L; < R; <5. Examples: 


1 is the word + Ea 1 
1,-1000(0:2) is the word -|1o00; 1 
-1000 (0:2),1 is the word + © 1 


10. The assembly process makes use of a value denoted by ® (called the 
location counter), which is initially zero. The value of ® should always be a 
nonnegative number that can fit in two bytes. When the location field of a line 
is not blank, it must contain a symbol that has not been previously defined. The 
equivalent of that symbol is then defined to be the current value of ®. 


11. After processing the LOC field as described in rule 10, the assembly 
process depends on the value of the OP field. There are six possibilities for OP: 


a) OP is a symbolic MIX operator (see Table 1 at the end of the previous section). 
The chart defines the normal C and F values for each MIX operator. In this 
case the ADDRESS should be an A-part (rule 6), followed by an index part 
(rule 7), followed by an F-part (rule 8). We thereby obtain four values: C, 
F, A, and I. The effect is to assemble the word determined by the sequence 
‘LDA C; STA WORD; LDA F; STA WORD(4:4); LDA I; STA WORD(3:3); LDA A; 
STA WORD(0:2)’ into the location specified by ®, and to advance ® by 1. 

b) OP is ‘EQU’. The ADDRESS should be a W-value (see rule 9). If the LOC field 
is nonblank, the equivalent of the symbol appearing there is set equal to the 
value specified in ADDRESS. This rule takes precedence over rule 10. The 
value of ® is unchanged. (As a nontrivial example, consider the line 


BYTESIZE EQU 1(4:4) 


which allows the programmer to have a symbol whose value depends on the 
byte size. This is an acceptable situation so long as the resulting program 
is meaningful with each possible byte size.) 

c) OP is ‘ORIG’. The ADDRESS should be a W-value (see rule 9); the location 
counter, ®, is set to this value. (Notice that because of rule 10, a symbol 
appearing in the LOC field of an ORIG line gets as its equivalent the value of 
® before it has changed. For example, 


TABLE ORIG *+100 


sets the equivalent of TABLE to the first of 100 locations.) 


d) OP is ‘CON’. The ADDRESS should be a W-value; the effect is to assemble a 
word, having this value, into the location specified by ®, and to advance ® 
by 1. 

e) OP is ‘ALF’. The effect is to assemble the word of character codes formed by 
the first five characters of the address field, otherwise behaving like CON. 
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f) OP is ‘END’. The ADDRESS should be a W-value, which specifies in its (4:5) 
field the location of the instruction at which the program begins. The END 
line signals the end of a MIXAL program. The assembler effectively inserts 
additional lines just before the END line, in arbitrary order, corresponding 
to all undefined symbols and literal constants (see rules 12 and 13). Thus a 
symbol in the LOC field of the END line will denote the first location following 
the inserted words. 

12. Literal constants: A W-value that is less than 10 characters long may 
be enclosed between ‘=’ signs and used as a future reference. The effect is to 
create a new symbol internally and to insert a CON line defining that symbol, 
just before the END line (see remark 4 following Program P). 


13. Every symbol has one and only one equivalent value; this is a full- 
word MIX number that is normally determined by the symbol’s appearance in 
LOC according to rule 10 or rule 11(b). If the symbol never appears in LOC, 
a new line is effectively inserted before the END line, having OP = ‘CON’ and 
ADDRESS = ‘0’ and the name of the symbol in LOC. 


Note: The most significant consequence of the rules above is the restriction 
on future references. A symbol that has not yet been defined in the LOC field 
of a previous line may not be used except as the A-part of an instruction. In 
particular, it may not be used (a) in connection with arithmetic operations; or 
(b) in the ADDRESS field of EQU, ORIG, or CON. For example, 


LDA 2F+1 


and 
CON 3F 


are both illegal. This restriction has been imposed in order to allow more efficient 
assembly of programs, and the experience gained in writing this set of books has 
shown that it is a mild limitation that rarely makes much difference. 


Actually MIX has two symbolic languages for low-level programming: MIXAL,* 
a machine-oriented language that is designed to facilitate one-pass translation 
by a very simple assembly program; and PL/MIX, which more adequately reflects 
data and control structures and which looks rather like the Remarks field of 
MIXAL programs. 


EXERCISES — First set 


1. [00] The text remarked that ‘X EQU 1000’ does not assemble any instruction that 
sets the value of a variable. Suppose that you are writing a MIX program in which 
the algorithm is supposed to set the value contained in a certain memory cell (whose 
symbolic name is X) equal to 1000. How could you express this in MIXAL? 

2. [10] Line 12 of Program M says ‘JMP *’, where * denotes the location of that line. 
Why doesn’t the program go into an infinite loop, endlessly repeating this instruction? 


* The author was astonished to learn in 1971 that MIXAL is also the name of a laundry 
detergent in Yugoslavia, developed for use with avtomate [automatics]. 
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> 3. [23] What is the effect of the following program, if it is used in conjunction with 
Program M? 


START IN X+1(0) 
JBUS *(0) 
ENT1 100 

1H JMP MAXIMUM 
LDX X,1 
STA X,1 
STX X,2 
DEC1 1 
JiP 1B 
OUT X+1(1) 
HLT 
END START J 


> 4. [25] Assemble Program P by hand. (It won’t take as long as you think.) What 
are the actual numerical contents of memory, corresponding to that symbolic program? 


5. [11] Why doesn’t Program P need a JBUS instruction to determine when the line 
printer is ready? 

6. [HM20] (a) Show that if n is not prime, n has a divisor d with 1 < d < y/n. (b) Use 
this fact to show that the test in step P7 of Algorithm P proves that N is prime. 


7. [10] (a) What is the meaning of ‘4B’ in line 34 of Program P? (b) What effect, if 
any, would be caused if the location of line 15 were changed to ‘2H’ and the address of 
line 20 were changed to ‘2B’? 


> 8. [24] What does the following program do? (Do not run it on a computer, figure 
it out by hand!) 


* MYSTERY PROGRAM 
BUF ORIG *+3000 
1H ENT1 1 
ENT2 0 
LDX 4F 
2H ENT3 0,1 
3H STZ BUF,2 
INC2 1 
DEC3 1 
J3P 3B 
STX BUF,2 
INC2 1 
INC1 1 
CMP1 =75= 
JL 2B 
ENN2 2400 
OUT BUF+2400,2(18) 
INC2 24 
JON *-2 
HLT 
4H ALF AAAAA 
END 1B J 


158 BASIC CONCEPTS 1.3.2 


EXERCISES — Second set 


These exercises are short programming problems, representing typical computer 
applications and covering a wide range of techniques. Every reader is encouraged to 
choose a few of these problems, in order to get some experience using MIX as well as 
a good review of basic programming skills. If desired, these exercises may be worked 
concurrently as the rest of Chapter 1 is being read. 

The following list indicates the types of programming techniques that are involved: 


The use of switching tables for multiway decisions: exercises 9, 13, and 23. 

The use of index registers with two-dimensional arrays: exercises 10, 21, and 23. 
Unpacking characters: exercises 13 and 23. 

Integer and scaled decimal arithmetic: exercises 14, 16, and 18. 

The use of subroutines: exercises 14 and 20. 

Input buffering: exercise 13. 

Output buffering: exercises 21 and 23. 

List processing: exercise 22. 

Real-time control: exercise 20. 

Graphical display: exercise 23. 


Whenever an exercise in this book says, “write a MIX program” or “write a MIX 
subroutine,” you need only write symbolic MIXAL code for what is asked. This code 
will not be complete in itself, it will merely be a fragment of a (hypothetical) complete 
program. No input or output need be done in a code fragment, if the data is to be 
supplied externally; one need write only LOC, OP, and ADDRESS fields of MIXAL lines, 
together with appropriate remarks. The numeric machine language, line number, and 
“times” columns (see Program M) are not required unless specifically requested, nor 
will there be an END line. 

On the other hand, if an exercise says, “write a complete MIX program,” it implies 
that an executable program should be written in MIXAL, including in particular the 
final END line. Assemblers and MIX simulators on which such complete programs can 
be tested are widely available. 


9. [25] Location INST contains a MIX word that purportedly is a MIX instruction. 
Write a MIX program that jumps to location GOOD if the word has a valid C-field, 
valid +AA-field, valid I-field, and valid F-field, according to Table 1.3.1—-1; your pro- 
gram should jump to location BAD otherwise. Remember that the test for a valid F-field 
depends on the C-field; for example, if C = 7 (MOVE), any F-field is acceptable, but if C 
= 8 (LDA), the F-field must have the form 8L +R where 0 < L < R < 5. The “tAA”- 
field is to be considered valid unless C specifies an instruction requiring a memory 
address and I = 0 and +AA is not a valid memory address. 


Note: Inexperienced programmers tend to tackle a problem like this by writing 
a long series of tests on the C-field, such as ‘LDA C; JAZ 1F; DECA 5; JAN 2F; JAZ 3F; 
DECA 2; JAN 4F; ...’. This is not good practice! The best way to make multiway 
decisions is to prepare an auxiliary table containing information that encapsulates the 
desired logic. If there were, for example, a table of 64 entries, we could write ‘LD1 C; 
LD1 TABLE,1; JMP 0,1’— thereby jumping very speedily to the desired routine. Other 
useful information can also be kept in such a table. A tabular approach to the present, 
problem makes the program only a little bit longer (including the table) and greatly 
increases its speed and flexibility. 


1.3.2 THE MIX ASSEMBLY LANGUAGE 159 


> 10. [31] Assume that we have a 9 x 8 matrix 


Q11 G12 Q13 ... G18 
Q21 Q22 Q23 ... Q28 
Q91 Q92 Q93 ... Q98 


stored in memory so that ai; is in location 1000+8i+j. In memory the matrix therefore 
appears as follows: 


(1009) (1010) (1011) ... (1016) 
(1017) (1018) (1019) ... (1024) 
(1073) (1074) (1075) ... (1080) 


A matrix is said to have a “saddle point” if some position is the smallest value in 
its row and the largest value in its column. In symbols, aij is a saddle point if 
Qij = min Qik = Max ag;. 
1<k<8 1<k<9 
Write a MIX program that computes the location of a saddle point (if there is at least 
one) or zero (if there is no saddle point), and stops with this value in rI1. 


11. [M29] What is the probability that the matrix in the preceding exercise has a 
saddle point, assuming that the 72 elements are distinct and assuming that all 72! 
arrangements are equally probable? What is the corresponding probability if we assume 
instead that the elements of the matrix are zeros and ones, and that all 27? such matrices 
are equally probable? 


12. [HM/2] Two solutions are given for exercise 10 (see page 512), and a third is 
suggested; it is not clear which of them is better. Analyze the algorithms, using each 
of the assumptions of exercise 11, and decide which is the better method. 


13. [28] A cryptanalyst wants a frequency count of the letters in a certain code. The 
code has been punched on paper tape; the end is signaled by an asterisk. Write a 
complete MIX program that reads in the tape, counts the frequency of each character 
up to the first asterisk, and then types out the results in the form 


A 0010257 
B 0000179 
D 0794301 


etc., one character per line. The number of blanks should not be counted, nor should 
characters for which the count is zero (like C in the above) be printed. For efficiency, 
“buffer” the input: While reading a block into one area of memory you can be counting 
characters from another area. You may assume that an extra block (following the one 
that contains the terminating asterisk) is present on the input tape. 


> 14. [31] The following algorithm, due to the Neapolitan astronomer Aloysius Lilius 
and the German Jesuit mathematician Christopher Clavius in the late 16th century, is 
used by most Western churches to determine the date of Easter Sunday for any year 
after 1582. 
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Algorithm E (Date of Easter). Let Y be the year for which the date of Easter is 
desired. 

E1. [Golden number.] Set G «+ (Y mod19)+1. (G is the so-called “golden 
number” of the year in the 19-year Metonic cycle.) 

E2. [Century.] Set C + |Y/100| + 1. (When Y is not a multiple of 100, C is the 
century number; for example, 1984 is in the twentieth century.) 

E3. [Corrections.] Set X + |3C/4| —12, Z + |(8C +5)/25| —5. (Here X is the 
number of years, such as 1900, in which leap year was dropped in order to 
keep in step with the sun; Z is a special correction designed to synchronize 
Easter with the moon’s orbit.) 

E4. [Find Sunday.] Set D + [5Y/4| —X—10. (March ((—D) mod 7) will actually 
be a Sunday.) 

E5. [Epact.] Set E + (11G + 20+ Z — X) mod 30. If E = 25 and the golden 
number G is greater than 11, or if Æ = 24, then increase E by 1. (This number 
E is the epact, which specifies when a full moon occurs.) 

E6. [Find full moon.] Set N + 44 — E. If N < 21 then set N + N +30. (Easter 
is supposedly the first Sunday following the first full moon that occurs on or 
after March 21. Actually perturbations in the moon’s orbit do not make this 
strictly true, but we are concerned here with the “calendar moon” rather than 
the actual moon. The Nth of March is a calendar full moon.) 

E7. [Advance to Sunday.] Set N + N+7—-((D+N)mod7). 


E8. [Get month.] If N > 31, the date is (N — 31) APRIL; otherwise the date is 
N MARCH. J 
Write a subroutine to calculate and print Easter date given the year, assuming 
that the year is less than 100000. The output should have the form “dd MONTH, yyyyy” 
where dd is the day and yyyyy is the year. Write a complete MIX program that uses 
this subroutine to prepare a table of the dates of Easter from 1950 through 2000. 


15. [M30] A fairly common error in the coding of the previous exercise is to fail to 
realize that the quantity (11G + 20+ Z — X) in step E5 may be negative; therefore the 
positive remainder mod 30 might not be computed properly. (See CACM 5 (1962), 
556.) For example, in the year 14250 we would find G = 1, X = 95, Z = 40; so if we 
had E = —24 instead of E = +6 we would get the ridiculous answer ‘42 APRIL’. Write 
a complete MIX program that finds the earliest year for which this error would actually 
cause the wrong date to be calculated for Easter. 


16. [31] We showed in Section 1.2.7 that the sum 1+ į +$ +--+- becomes infinitely 
large. But if it is calculated with finite accuracy by a computer, the sum actually 
exists, in some sense, because the terms eventually get so small that they contribute 
nothing to the sum if added one by one. For example, suppose we calculate the sum 
by rounding to one decimal place; then we have 1 + 0.5 + 0.3 + 0.3 + 0.2 + 0.2 + 0.1 4 
0.1 + 0.1 +0.1+0.1+0.1+0.1+0.1+0.1 +0.1 +0.1 +0.1 +0.1 + 0.1 = 3.9. 

More precisely, let rn(x) be the number x rounded to n decimal places; we define 
Ta(x) = [10"2 + 4|/10”. Then we wish to find 


Sn =Tn(1) +ra(S) + ra(Z) +; 


we know that S; = 3.9, and the problem is to write a complete MIX program that 
calculates and prints Sn for n = 2, 3, 4, and 5. 
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Note: There is a much faster way to do this than the simple procedure of adding 
rn(1/m), one number at a time, until rn(1/m) becomes zero. For example, we have 
rs5(1/m) = 0.00001 for all values of m from 66667 to 200000; it’s wise to avoid 
calculating 1/m all 133334 times! An algorithm along the following lines should rather 
be used: 

A. Start with mpn = 1, S=1. 

B. Set me = Mp +1 and calculate r,(1/m_) =r. 

C. Find mp, the largest m for which r,,(1/m) = r. 

D. Add (Mn — me + 1)r to S and return to Step B. 


17. [HM30] Using the notation of the preceding exercise, prove or disprove the formula 
limn—oo(Sn+1 — Sn) = In 10. 


18. [25] The ascending sequence of all reduced fractions between 0 and 1 that have 
denominators < n is called the “Farey series of order n.” For example, the Farey series 
of order 7 is 


0 1 1 1 1 2 1 2 3 1 43253456 1 

1’ 7 6’ 5’ 4’ 7’ 3’? 5’ 7’ 2’ TES 7 4° 5° 6 7’ 1 

If we denote this series by xo/yo, £1/Y1, £2/Y2, ..., exercise 19 proves that 
ro =0, y= l; m=1l, y =n; 


Tk+2 = | (yk + n)/Yk+1]Tk+1 — Tk; 


Yk+2 = Lye + n)/yk+1]Yk+ı — Yk- 


Write a MIX subroutine that computes the Farey series of order n, by storing the values 
of £k and yx in locations X + k, Y + k, respectively. (The total number of terms in the 
series is approximately 3n?/n?, so you may assume that n is rather small.) 


19. [M30] (a) Show that the numbers xp and yx defined by the recurrence in the 
preceding exercise satisfy the relation £k+1Yk —TkYk+ı = 1. (b) Show that the fractions 
xk /Yyr are indeed the Farey series of order n, using the fact proved in (a). 


> 20. [33] Assume that MIX’s overflow toggle and X-register have been wired up to the 
traffic signals at the corner of Del Mar Boulevard and Berkeley Avenue, as follows: 


rX(2:2) = Del Mar traffic light f 
rX(3:3) = Berkeley traffic light Oof Tigreen, 2 ambet, 3 red, 
rX(4:4) = Del Mar pedestrian light P P SNAS P 
rX(5:5) = Berkeley pedestrian light Oot, ee DONT T WALK? 


Cars or pedestrians wishing to travel on Berkeley across the boulevard must trip a 
switch that causes the overflow toggle of MIX to go on. If this condition never occurs, 
the light for Del Mar should remain green. 

Cycle times are as follows: 


Del Mar traffic light is green > 30 sec, amber 8 sec; 
Berkeley traffic light is green 20 sec, amber 5 sec. 


When a traffic light is green or amber for one direction, the other direction has a red 
light. When the traffic light is green, the corresponding WALK light is on, except that 
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22 | 47 | 16 | 41 | 10 | 35 | 04 
05 | 23 | 48 | 17 | 42 | 11 | 29 
30 | 06 | 24 | 49 | 18 | 36 | 12 
13 | 31 | 07 | 25 | 43 | 19 | 37 
38 | 14 | 32 | O1 | 26 | 44 | 20 
21 | 39 | 08 | 33 | 02 | 27 | 45 
46 | 15 | 40 | 09 | 34 | 03 | 28 


Fig. 16. A magic square. Fig. 17. Josephus’s problem, n = 8, m = 4. 


DON’T WALK flashes for 12 sec just before a green light turns to amber, as follows: 


DON’T WALK 
off 2 
DON’T WALK 4 sec (and remains on through amber and red cycles). 


4 ec , 
i repeat 8 times; 
= sec 


If the overflow is tripped while the Berkeley light is green, the car or pedestrian 
will pass on that cycle, but if it is tripped during the amber or red portions, another 
cycle will be necessary after the Del Mar traffic has passed. 

Assume that one MIX time unit equals 10 usec. Write a complete MIX program 
that controls these lights by manipulating rX, according to the input given by the 
overflow toggle. The stated times are to be followed exactly unless it is impossible to 
do so. Note: The setting of rX changes precisely at the completion of a LDX or INCX 
instruction. 


21. [28] A magic square of order n is an arrangement of the numbers 1 through n? in 
a square array in such a way that the sum of each row and column is n(n?+1)/2, and so 
is the sum of the two main diagonals. Figure 16 shows a magic square of order 7. The 
rule for generating it is easily seen: Start with 1 just below the middle square, then go 
down and to the right diagonally —when running off the edge imagine an entire plane 
tiled with squares— until reaching a filled square; then drop down two spaces from the 
most-recently-filled square and continue. This method works whenever n is odd. 

Using memory allocated in a fashion like that of exercise 10, write a complete 
MIX program to generate the 23 x 23 magic square by the method above, and to print 
the result. [This algorithm is due to Ibn al-Haytham, who was born in Basra about 
965 and died in Cairo about 1040. Many other magic square constructions make good 
programming exercises; see W. W. Rouse Ball, Mathematical Recreations and Essays, 
revised by H. S. M. Coxeter (New York: Macmillan, 1939), Chapter 7.] 


22. [31] (The Josephus problem.) There are n men arranged in a circle. Beginning at 
a particular position, we count around the circle and brutally execute every mth man; 
the circle closes as men die. For example, the execution order when n = 8 and m = 4 is 
54613872, as shown in Fig. 17: The first man is fifth to go, the second man is fourth, 
etc. Write a complete MIX program that prints out the order of execution when n = 24, 
m = 11. Try to design a clever algorithm that works at high speed when n and m are 
large (it may save your life). Reference: W. Ahrens, Mathematische Unterhaltungen 
und Spiele 2 (Leipzig: Teubner, 1918), Chapter 15. 
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23. [37] This is an exercise designed to give some experience in the many applications 
of computers for which the output is to be displayed graphically rather than in the usual 


tabular form. In this case, the object is to “draw” 


You are given as input a matrix of zeros and 
ones. An entry of zero indicates a white square; a 
one indicates a black square. The output should bea 
diagram of the puzzle, with the appropriate squares 
numbered for words across and down. 

For example, given the matrix 


10000 1 
0 01 0 0 0 
00 00 1 0 
0 100 0 07)’ 
00 01 0 0 
1000 0 1 


a crossword puzzle diagram. 


1 2 13 


CT 


Fig. 18. Diagram corresponding 
to the matrix in exercise 23. 


the corresponding puzzle diagram would be as shown in Fig. 18. A square is numbered 
if it is a white square and either (a) the square below it is white and there is no white 
square immediately above, or (b) the square to its right is white and there is no white 
square immediately to its left. If black squares occur at the edges, they should be 
removed from the diagram. This is illustrated in Fig. 18, where the black squares at 
the corners were dropped. A simple way to accomplish this is to artificially insert rows 
and columns of —1’s at the top, bottom, and sides of the given input matrix, then to 
change every +1 that is adjacent to a —1 into a —1 until no +1 remains next to any —1. 

The following method should be used to print the final diagram on a line printer: 
Each box of the puzzle should correspond to 5 columns and 3 rows of the output page, 


where the 15 positions are filled as follows: 


Unnumbered wut Number nn nnyyj+ Black +4444 
white squares:  uuuut white squares: Qyuut squares: +++++ 
+++++ +++++ +++++ 


“—1” squares, depending on whether there are —1’s to the right or below: 


uuuut uuuut WWIII 
uuuut uuuut WWIII 
+++++ Haai +++++ 


The diagram shown in Fig. 18 would then be printed 


The width of a printer line — 120 characters — 
is enough to allow up to 23 columns in the crossword 
puzzle. The data supplied as input to your program 
will be a 23 x 23 matrix of zeros and ones, where 
each row is punched in columns 1-23 of an input 
card. For example, the card corresponding to the 
top row of the matrix above would be punched 
‘10000111111111111111111’. The diagram will not 
necessarily be symmetrical, and it might have long 
paths of black squares that are connected to the 
outside in strange ways. 


Fig. 19. Representation of Fig. 18 on a line printer. 


Uuuu Huu 
HULL Huu 
out HUW 


as shown in Fig. 19. 


FHEEFEFTEFEPEPE EEE 


+01 + +02 +03 + 

+ + + + + 
FHFFFAFTFTEFEFEEEEPEPTET EEE EET 
+04 + ++++++05 + +06 + 
+ + ++++++ + + + 
HEHHHHHHHHHHH+HH+HH+H+HH+H+H+H+H+H+H+H+H+H+H+ 
+07 + +08 + ++++++ + 
+ + + + ++++++ + 
HEHHHEEHHHHHHH+HH+HH+H+H+H+H+H+H+H+H+H+H+H+H+H+ 
+ ++++++09 + +10 + + 
+ ++++++ + + + + 
HEHHHHHHHHHHH+HH+HH+H+H+H+H+H+H+H+H+H+H+H+H+H+ 
+11 +12 + ++++++13 + + 
+ + + ++++++ + + 
HEHHHHHHHHHHH+HH+HH+H+HH+H+H+H+H+H+H+H+H+H+H+ 

+14 + + + + 


+ + + + + 
HHHHHHH+H+H+H+H+H+H+H+H+H+H+H+H+H+ 
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1.3.3. Applications to Permutations 


In this section we shall give several more examples of MIX programs, and at the 
same time introduce some important properties of permutations. These inves- 
tigations will also bring out some interesting aspects of computer programming 
in general. 

Permutations were discussed earlier in Section 1.2.5; we treated the per- 
mutation cdfbea as an arrangement of the six objects a, b, c, d, e, f ina 
straight line. Another viewpoint is also possible: We may think of a permutation 
as a rearrangement or renaming of the objects. With this interpretation it is 
customary to use a two-line notation, for example, 


abcdef 
(° df be ay (1) 
to mean “a becomes c, b becomes d, c becomes f, d becomes b, e becomes e, 
f becomes a.” Considered as a rearrangement, this means that object c moves 
to the place formerly occupied by object a; considered as a renaming, it means 


that object a is renamed c. The two-line notation is unaffected by changes in 
the order of the columns; for example, the permutation (1) could also be written 


cdfbae 
fbadce 
and in 718 other ways. 


A cycle notation is often used in connection with this interpretation. Permu- 
tation (1) could be written 


(ac f) (bd), (2) 
again meaning “a becomes c, c becomes f, f becomes a, b becomes d, d be- 
comes b.” A cycle (£1 £2... £n) means “x, becomes T2, ..., Lp_1 becomes £n, 


” 


£n becomes x1.” Since e is fixed under the permutation, it does not appear 

in the cycle notation; that is, singleton cycles like “(e)” are conventionally not 

written. If a permutation fixes all elements, so that there are only singleton 

cycles present, it is called the identity permutation, and we denote it by “()”. 
The cycle notation is not unique. For example, 


(bd) (acf), (cfa) (bd), — (db) (fac), (3) 


etc., are all equivalent to (2). However, “(a f c) (b d)” is not the same, since it 
says that a goes to f. 

It is easy to see why the cycle notation is always possible. Starting with 
any element z1, the permutation takes zı into £2, say, and x9 into z3, etc., until 
finally (since there are only finitely many elements) we get to some element £n+1 
that has already appeared among z£1,..., £n. Now 2+ 1 must equal xı. For if it 
were equal to, say, x3, we already know that x2 goes into x3; but by assumption, 
Ln Æ LQ goes tO n1: SO Lp41 = T1, and we have a cycle (£1 £2... Ln) as 
part of our permutation, for some n > 1. If this does not account for the entire 
permutation, we can find another element yı and get another cycle (y1 yo... Ym) 
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in the same way. None of the y’s can equal any of the x’s, since z; = y; implies 
that 2441 = yj+1, etc., and we would ultimately find 2, = yı for some k, 
contradicting the choice of y1. All cycles will eventually be found. 

One application of these concepts to programming comes up whenever some 
set of n objects is to be put into a different order. If we want to rearrange 
the objects without moving them elsewhere, we must essentially follow the cycle 
structure. For example, to do the rearrangement (1), namely to set 


(a, b, c, d, e, f) e (c, d, f: b, e, a), 


we would essentially follow the cycle structure (2) and successively set 


tea, atc, ctf, fet teb, bed, det. 


It is frequently useful to realize that any such transformation takes place in 
disjoint cycles. 


Products of permutations. We can multiply two permutations together, with 
the understanding that multiplication means the application of one permutation 
after the other. For example, if permutation (1) is followed by the permutation 


abcdef 
bdcafe/)’ 


we have a becomes c, which then becomes c; b becomes d, which becomes a; etc.: 


abcdef abcdef 
cdfbea * bdcafe 


It should be clear that multiplication of permutations is not commutative; 
in other words, 7 X T2 is not necessarily equal to 72 x mı when 71 and m are 
permutations. The reader may verify that the product in (4) gives a different 
result if the two factors are interchanged (see exercise 3). 

Some people multiply permutations from right to left rather than the some- 
what more natural left-to-right order shown in (4). In fact, mathematicians are 
divided into two camps in this regard; should the result of applying transforma- 
tion T}, then Ty, be denoted by 7,75 or by ToT? Here we use TTo. 

Equation (4) would be written as follows, using the cycle notation: 


(ac f) (bd) (abd) (e f) = (ace f b). (5) 


Note that the multiplication sign “x” is conventionally dropped; this does not 
conflict with the cycle notation since it is easy to see that the permutation 
(ac f)(bd) is really the product of the permutations (ac f) and (bd). 
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Multiplication of permutations can be done directly in terms of the cycle 
notation. For example, to compute the product of several permutations 


(ac fg) (bed) (aed) (fade) (bg fae), (6) 


we find (proceeding from left to right) that “a goes to c, then c goes to d, then 
d goes to a, then a goes to d, then d is unchanged”; so the net result is that 
a goes to d under (6), and we write down “(ad” as the partial answer. Now 
we consider the effect on d: “d goes to b goes to g”; we have the partial result 
“(adg”. Considering g, we find that “g goes to a, to e, to f, to a”, and so the 
first cycle is closed: “(adg)”. Now we pick a new element that hasn’t appeared 
yet, say c; we find that c goes to e, and the reader may verify that ultimately 
the answer “(adg)(ceb)” is obtained for (6). 

Let us now try to do this process by computer. The following algorithm 
formalizes the method described in the preceding paragraph, in a way that is 
amenable to machine calculation. 


a > matched 

| A3. Set CURRENT }+—> A4. Scan formula 

Al. First pass -—> A2. Open ae | Mj F 
Done N | he. Close kas. CURRENT = START? ) 


Fig. 20. Algorithm A for multiplying permutations. 


Algorithm A (Multiply permutations in cycle form). This algorithm takes a 
product of cycles, such as (6), and computes the resulting permutation in the 
form of a product of disjoint cycles. For simplicity, the removal of singleton cycles 
is not described here; that would be a fairly simple extension of the algorithm. 
As this algorithm is performed, we successively “tag” the elements of the input 
formula; that is, we mark somehow those symbols of the input formula that have 
been processed. 


A1. [First pass.] Tag all left parentheses, and replace each right parenthesis by 
a tagged copy of the input symbol that follows its matching left parenthesis. 
(See the example in Table 1.) 


A2. [Open.] Searching from left to right, find the first untagged element of the 
input. (If all elements are tagged, the algorithm terminates.) Set START 
equal to it; output a left parenthesis; output the element; and tag it. 


A3. [Set CURRENT.] Set CURRENT equal to the next element of the formula. 


A4. [Scan formula.] Proceed to the right until either reaching the end of the 
formula, or finding an element equal to CURRENT; in the latter case, tag it 
and go back to step A3. 
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Table 1 

ALGORITHM A APPLIED TO (6) 
After 
step START CURRENT (acfg)(bcd)(aed)(fade)(bgfae) Output 
Al acfg bed aed fade bgfae 
A2 a le fg bed aed fade bgfae (a 
A3 a c elf 9 bed aed fade bgfae 
A4 a c cfg b Ja aed fade bgfae 
A4 a d cfg bcd aedi fade bgfae 
A4 a a cfg bcd ae falde bgfae 
A5 a d cfg bcd ae fade bgfaeb] d 
A5 a g cfg b ae fade gfaeb] g 
A5 a a cf b e fad gfaedbl 
A6 a a cf b e fad gfaebl ) 
A2 c a Tf b e fad gjae (c 
A5 c e f b e d g e b| e 
A5 c b f b g T è 
A6 c c f g I) 
A6 f f I (f) 


Here | represents a cursor following the element just scanned; tagged elements are light gray. 


A5. [CURRENT = START?] If CURRENT Æ START, output CURRENT and go back to 
step A4 starting again at the left of the formula (thereby continuing the 
development of a cycle in the output). 


A6. [Close.] (A complete cycle in the output has been found.) Output a right 
parenthesis, and go back to step A2. I 


For example, consider formula (6); Table 1 shows successive stages in its 
processing. The first line of that table shows the formula after right parentheses 
have been replaced by the leading element of the corresponding cycle; succeeding 
lines show the progress that is made as more and more elements are tagged. 
A cursor shows the current point of interest in the formula. The output is 
“(adg) (ceb) (f)”; notice that singleton cycles will appear in the output. 


A MIX program. To implement this algorithm for MIX, the “tagging” can be 
done by using the sign of a word. Suppose our input is punched onto cards in 
the following format: An 80-column card is divided into 16 five-character fields. 
Each field is either (a) ‘uuusC, representing the left parenthesis beginning a 
cycle; (b) ‘) uuu’, representing the right parenthesis ending a cycle; (c) ‘uuu’, 
all blanks, which may be inserted anywhere to fill space; or (d) anything else, 
representing an element to be permuted. The last card of the input is recognized 
by having columns 76-80 equal to ‘4uuu=’. For example, (6) might be punched 
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on two cards as follows: 


(A C FEF G) ( B C D) (A E D) 
( F A D E) ( B G F A E) = 


The output of our program will consist of a verbatim copy of the input, followed 
by the answer in essentially the same format. 


Program A (Multiply permutations in cycle form). This program implements 
Algorithm A, and it also includes provision for input, output, and the removing 
of singleton cycles. But it doesn’t catch errors in the input. 


01 MAXWDS EQU 1200 Maximum length of input 

02 PERM ORIG *+MAXWDS The input permutation 

03 ANS ORIG *+MAXWDS Place for answer 

04 QUTBUF ORIG *+24 Place for printing 

05 CARDS EQU 16 Unit number for card reader 
06 PRINTER EQU 18 Unit number for printer 

07 BEGIN IN  PERM(CARDS) Read first card. 

08 ENT2 0 

09 LDA EQUALS 

10 1H JBUS *(CARDS) Wait for cycle complete. 

11 CMPA PERM+15,2 

12 JE *+2 Is it the last card? 

18 IN PERM+16,2(CARDS) No, read another. 

14 ENT1 OUTBUF 

15 JBUS *(PRINTER) Print a copy of 

16 MOVE PERM,2(16) the input card. 

17 OUT OUTBUF (PRINTER) 

18 JE 1F 

19 INC2 16 

20 CMP2 =MAXWDS-16= 

21 JLE 1B Repeat until input is complete. 
22 HLT 666 Too much input! 

23 1H INC2 15 1 At this point, rl2 words of 

24 ST2 SIZE 1 input are in PERM, PERM + 1, ... 
25 ENT3 O 1 Al. First pass. 

26 2H LDAN PERM,3 A Get next element of input. 

27 CMPA LPREN(1:5) A Isit‘? 

28 JNE 1F A 

29 STA PERM,3 B If so, tag it. 

30 INC3 1 B Put the next nonblank input symbol 
31 LDXN PERM,3 B into rX. 

32 JXZ *-2 B 

33 1H CMPA RPREN(1:5) C 

34 JNE *+2 C 

35 STX PERM,3 D Replace ‘)’ by tagged rX. 

36 INC3 1 C 

37 CMP3 SIZE C Have all elements been processed? 
38 JL 2B C 


OPEN 
1H 


DONE 


* 
LPREN 
RPREN 
EQUALS 
* 


GO 


SUCC 


5H 


4H 


1H 


CLOSE 


SIZE 
1B 


=ANS= 
*+2 
LPREN (2) 
=0= 
-1,1(22) 
0 


ANS ,3 (PRINTER) 


24 
ANS,3 
*-3 


LPREN 
PERM,3 
START 
PERM,3 

1 
PERM,3(1:5) 
1F 

*-3 

0,1 

1 

0 

PERM ,3(1:5) 
SUCC 

1 

SIZE 

4B 
START(1:5) 
5B 

RPREN 

-3,1 

OPEN 

-3 

OPEN 

BEGIN 
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QQA DARE 


HNDDDDVEHDEARAOOO UUW 
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Prepare for main program. 

rll = place to store next answer 
A2. Open. 

Look for untagged element. 


All are tagged. Now comes the output. 


Is answer the identity permutation? 
If so, change to ‘()’. 
Put 23 words of blanks after answer. 


Print as many lines as necessary. 


Constants used in the program 


Open a cycle in the output. 


Tag an element. 

Move one step to the right. 
A3. Set CURRENT (namely rX). 
Skip past blanks. 


Output CURRENT. 


Scan formula again. 
A4. Scan formula. 
Element = CURRENT? 
Move to right. 

End of formula? 


A5. CURRENT = START? 


A6. Close. 
Note: rA = ‘C. 


Suppress singleton cycles. 
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This program of approximately 75 instructions is quite a bit longer than 
the programs of the previous section, and indeed it is longer than most of the 
programs we will meet in this book. Its length is not formidable, however, since 
it divides into several small parts that are fairly independent. Lines 07-22 read 
in the input cards and print a copy of each card; lines 23-38 accomplish step Al 
of the algorithm, the preconditioning of the input; lines 39-46 and 64-86 do the 
main business of Algorithm A; and lines 48-57 output the answer. 

The reader will find it instructive to study as many of the MIX programs given 
in this book as possible. An ability to read and to understand computer programs 
that you haven’t written yourself is exceedingly important; yet such training has 
been sadly neglected in too many computer courses, and some horribly inefficient 
uses of computing machinery have arisen as a result. 


Timing. The parts of Program A that are not concerned with input-output 
have been decorated with frequency counts, as we did for Program 1.3.2M. Thus, 
for example, line 30 is supposedly executed B times. For convenience we shall 
assume that no blank words appear in the input except at the extreme right end; 
under this assumption, line 71 is never executed and the jump in line 32 never 
occurs. 

By simple addition the total time to execute the program is 


(7+5A+6B+7C+2D+E+3F+4G+8H+6J 
+3K +4L+3P+4Q+6R+25)u, (7) 


plus the time for input and output. In order to understand the meaning of 
formula (7), we need to examine the fifteen unknowns A, B, C, D, E, F, G, H, 
J, K, L, P, Q, R, S and we must relate them to pertinent characteristics of the 
input. Let’s look at some general principles of attack for problems of this kind. 

First we can apply “Kirchhoff’s first law” of electrical circuit theory: The 
number of times an instruction is executed must equal the number of times we 
transfer to that instruction. This seemingly obvious rule often relates several 
quantities in a nonobvious way. Analyzing the flow of Program A, we get the 
following equations. 


From lines We deduce 

26, 38, 39 A=1+4+(C-1) 

33, 28, 29 C=B+(A-B) 

41, 84, 86 E=1+R 

42, 46, 48 F=E+(G-1) 

64, 43, 44 H=F-G 

67, 70, 76 J=H+(K-—(L-—J)) 
75, 79, 80 K=Q+(L-P) 


82,72,81 R=P-Q 


The equations given by Kirchhoff’s law will not all be independent; in the present 
case, for example, we see that the first and second equations are obviously 
equivalent. Furthermore, the last equation can be deduced from the others, 
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since the third, fourth, and fifth imply that H = R; hence the sixth says that 
K = L- R. At any rate we have already eliminated six of our fifteen unknowns: 


A=C, E=R+1, F=R+G, H=R, K=L-R, Q=P-R. (8) 


Kirchhoff’s first law is an effective tool that is analyzed more closely in Section 
234.1. 

The next step is to try to match up the variables with important character- 
istics of the data. We find from lines 24, 25, 30, and 36 that 


B + C = number of words of input = 16X — 1, (9) 


where X is the number of input cards. From line 28, 


B = number of “(” in input = number of cycles in input. (10) 
Similarly, from line 34, 
D = number of “)” in input = number of cycles in input. (11) 


Now (10) and (11) give us a fact that could not be deduced by Kirchhoff’s law: 
B=D. (12) 

From line 64, 
H = number of cycles in output (including singletons). (13) 


Line 82 says R is equal to this same quantity; the fact that H = R was in this 
case deducible from Kirchhoff’s law, since it already appears in (8). 
Using the fact that each nonblank word is ultimately tagged, and lines 29, 
35, and 67, we find that 
J=Y —2B, (14) 


where Y is the number of nonblank words appearing in the input permutations. 
From the fact that every distinct element appearing in the input permutation is 
written into the output just once, either at line 65 or line 72, we have 


P = H + Q = number of distinct elements in input. (15) 


(See Eqs. (8).) A moment’s reflection makes this clear from line 80 as well. 
Finally, we see from line 85 that 


S = number of singleton cycles in output. (16) 


Clearly the quantities B, C, H, J, P, and S that we have now interpreted 
are essentially independent parameters that may be expected to enter into the 
timing of Program A. 

The results we have obtained so far leave us with only the unknowns G 
and L to be analyzed. For these we must use a little more ingenuity. The scans 
of the input that start at lines 41 and 74 always terminate either at line 47 (the 
last time) or at line 80. During each one of these P + 1 loops, the instruction 
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‘INC3 1’ is performed B+ C times; this takes place only at lines 44, 68, and 77, 
so we get the nontrivial relation 


G+J+L=(B+C)(P+1) (27) 


connecting our unknowns G and L. Fortunately, the running time (7) is a func- 
tion of G+ L (it involves ---+3F+4G+---+3K+4L+---=---+7G+7L+---), 
so we need not try to analyze the individual quantities G and L any further. 

Summing up all these results, we find that the total time exclusive of input- 
output comes to 


(112NX + 304X — 2M — Y + 11U + 2V — 11)u; (18) 
in this formula, new names for the data characteristics have been used as follows: 


X = number of cards of input, 

Y = number of nonblank fields in input (excluding final “=”), 
M = number of cycles in input, 

N = number of distinct element names in input, 

U = number of cycles in output (including singletons), 

V = number of singleton cycles in output. 


In this way we have found that analysis of a program like Program A is in many 
respects like solving an amusing puzzle. 

We will show below that, if the output permutation is assumed to be random, 
the quantities U and V will be Hy and 1, respectively, on the average. 


Another approach. Algorithm A multiplies permutations together much as 
people ordinarily do the same job. Quite often we find that problems to be solved 
by computer are very similar to problems that have confronted humans for many 
years; therefore time-honored methods of solution, which have evolved for use 
by mortals such as we, are also appropriate procedures for computer algorithms. 

Just as often, however, we encounter new methods that turn out to be 
superior for computers, although they are quite unsuitable for human use. The 
central reason is that a computer “thinks” differently; it has a different kind of 
memory for facts. An instance of this difference may be seen in our permutation- 
multiplication problem: Using the algorithm below, a computer can do the 
multiplication in one sweep over the formula, remembering the entire current 
state of the permutation as its cycles are being multiplied. The human-oriented 
Algorithm A scans the formula many times, once for each element of the output, 
but the new algorithm handles everything in one scan. This is a feat that could 
not be done reliably by Homo sapiens. 

What is this computer-oriented method for permutation multiplication? 
Table 2 illustrates the basic idea. The column below each character of the cycle 
form in that table says what permutation is represented by the partial cycles to 
the right. For example, the fragmentary formula “... d e)(b g f ae)” represents 
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Table 2 
MULTIPLYING PERMUTATIONS IN ONE PASS 


(acfg) (bcd) (aed) (fade) (bgfae) 


a—+ddaaaaaaaaaaaddddddeceeececeaa 


b+ ccecccecceggggggggggggggggbbbbb 
c>eeeddddddcccccccecccceccceccce 
d>ggggg99gg9)))dd)))bbbbbddddddddd 
e>bbbbbbbbbbbbbbaaa))))bb)))))e 


foffffeeeeeceeceeececeaaaaaaaaff f 


g—>a))))FFFFFFFFFFFEFFFFEFFFFF9999 


abcdefg 
egcb? af)?’ 


which appears under the rightmost d of the table, except that the unknown 
destination of e is represented there by ‘)’ not ‘?’. 

Inspection of Table 2 shows that it can be created systematically, if we 
start with the identity permutation on the right and work backward from right 
to left. The column below letter x differs from the column to its right (which 
records the previous status) only in row x; and the new value in row z is the one 
that disappeared in the preceding change. More precisely, we have the following 
algorithm: 


the permutation 


Algorithm B (Multiply permutations in cycle form). This algorithm accom- 

plishes essentially the same result as Algorithm A. Assume that the elements per- 

muted are named 21, %2,...,%p. We use an auxiliary table T[1], T[2],...,7[nJ; 

upon termination of this algorithm, x; goes to x; under the input permutation 

if and only if T[i] = j. 

B1. [Initialize.] Set T[k] + k for 1 < k < n. Also, prepare to scan the input 
from right to left. 

B2. [Next element.] Examine the next element of the input (right to left). If 
the input has been exhausted, the algorithm terminates. If the element is a 
“\” set Z + 0 and repeat step B2; if it is a “(”, go to B4. Otherwise the 
element is x; for some 7; go on to B3. 

B3. [Change T|?].] Exchange Z © T[i]. If this makes T[i] = 0, set j 4+ i. Return 
to step B2. 

B4. [Change T[j].] Set T[j] + Z. (At this point, j is the row that shows a “)” 
entry in the notation of Table 2, corresponding to the right parenthesis that 
matches the left parenthesis just scanned.) Return to step B2. | 


Of course, after this algorithm has been performed, we still must output the 
contents of table T in cycle form; this is easily done by a “tagging” method, as 
we shall see below. 
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| 


B1. Initialize 


i D 


B3. Change Ti] | B4. Change T[j] 


ri 


Fig. 21. Algorithm B for multiplying permutations. 


Let us now write a MIX program based on the new algorithm. We wish to use 
the same ground rules as those in Program A, with input and output in the same 
format as before. A slight problem presents itself; namely, how can we implement 
Algorithm B without knowing in advance what the elements 21, £2,..., 2n are? 
We don’t know n, and we don’t know whether the element named b is to be 
z1, Or £2, etc. A simple way to solve this problem is to maintain a table of the 
element names that have been encountered so far, and to search for the current 
name each time (see lines 35-44 in the program below). 


Program B (Same effect as Program A). rX = Z; rl4 = i; rll = j; rI3 = n, 
the number of distinct names seen. 


01 MAXWDS EQU 1200 Maximum length of input 
02 X ORIG *+MAXWDS The table of names 

03 T ORIG *+MAXWDS The auxiliary state table 
04 PERM ORIG *+MAXWDS The input permutation 
05 ANS EQU PERM Place for answer 

06 OUTBUF ORIG *+24 Place for printing 


07 CARDS EQU 16 
e Same as lines 05-22 of Program A 
24 HLT 666 At this point, r12 words of 


25 1H INC2 15 1 input are in PERM, PERM+1,... 
26 ENT3 O 1 and we haven’t seen any names yet. 
27 RIGHT ENTX O A Set Z7< 0. 

28 SCAN ODEC2 1 B B2. Next element. 

29 LDA PERM,2 B 

30 JAZ CYCLE B Skip over blanks. 

31 CMPA RPREN C 

32 JE RIGHT C Is the next element “)”? 

33 CMPA LPREN D 

34 JE LEFT D Isit“(”? 

35 ENT4 1,3 E Prepare for the search. 

36 STA X E Store at beginning of table. 

87 2H DEC4 1 F Search through names table. 

38 CMPA X,4 F 

39 JNE 2B F Repeat until match found. 

40 J4P FOUND G Has the name appeared before? 
41 INC3 1 H No; increase the table size. 

42 STA X,3 H Insert the new name Zn. 
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43 ST3 T,3 H Set T[n] Hn, 

44 ENT4 0,3 H ten, 

45 FOUND LDA T,4 J B3. Change T{i]. 

46 STX T,4 J Store Z. 

47 SRC 5 J Set Z. 

48 JANZ SCAN J 

49 ENT1 0,4 K If Z was zero, set j + i. 

50 JMP SCAN K 

51 LEFT STX T,1 L B4. Change T|j]. 

52 CYCLE J2P SCAN P Return to B2, unless finished. 
539 * 

54 OUTPUT ENT1 ANS 1 All input has been scanned. 
55 J3Z DONE 1 The zx and T tables contain the answer. 
56 1H LDAN X,3 Q Now we construct cycle notation. 
57 JAP SKIP Q Has name been tagged? 

58 CMP3 T,3 R Is there a singleton cycle? 

59 JE SKIP R 

60 MOVE LPREN S Open a cycle. 

61 2H MOVE X,3 T 

62 STA X,3 T Tag the name. 

68 LD3 T,3 T Find successor of element. 

64 LDAN X,3 T 

65 JAN 2B T Isit already tagged? 

66 MOVE RPREN W Yes, cycle closes. 

67 SKIP DEC3 1 Z Move to next name. 

68 J3P 1B Z 

69 * 


70 DONE CMP1 =ANS= 
sars Same as lines 48-62 of Program A 

84 EQUALS ALF = 

85 END BEGIN I 


Lines 54-68, which construct the cycle notation from the T table and the table 
of names, make a rather pretty little algorithm that merits some study. The 
quantities A, B, ..., R, S, T, W, Z that enter into the timing of this program 
are, of course, different from the quantities of the same name in the analysis of 
Program A. The reader will find it an interesting exercise to analyze these times 
(see exercise 10). 

Experience shows that the main portion of the execution time of Program B 
will be spent in searching the names table—this is quantity F in the timing. 
Much better algorithms for searching and building dictionaries of names are 
available; they are called symbol table algorithms, and they are of great impor- 
tance in computer applications. Chapter 6 contains a thorough discussion of 
efficient symbol table algorithms. 


Inverses. The inverse m~ of a permutation 7 is the rearrangement that undoes 
the effect of m; if i goes to j under 7, then 7 goes to i under m~. Thus the 
product mz” equals the identity permutation, and so does the product m~r. 
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People often denote the inverse by m~! instead of m7, but the superscript 1 is 
redundant (for the same reason that x! = x). 

Every permutation has an inverse. For example, the inverse of 


abcdef ; cdfbea\ (abcdef 
Ge $ Geet 
We will now consider some simple algorithms for computing the inverse of a 
permutation. 

In the rest of this section, let us assume that we are dealing with permuta- 
tions of the numbers {1,2,...,n}. If X[1] X[2]...X[n] is such a permutation, 
there is a simple method to compute its inverse: Set Y[X[k]] — k forl <k <n. 
Then Y[1] Y [2]... Y [n] is the desired inverse. This method uses 2n memory cells, 
namely n for X and n for Y. 

Just for fun, however, let’s suppose that n is very large and suppose also 
that we wish to compute the inverse of X[1] X[2]...X[n] without using much 
additional memory space. We want to compute the inverse “in place,” so that 
after our algorithm is finished the array X[1] X[2]...X|n] will be the inverse 
of the original permutation. Merely setting X[X|k]] +} k for 1 < k < n will 
certainly fail, but by considering the cycle structure we can derive the following 
simple algorithm: 


Algorithm I (Inverse in place). Replace X[1]X[2]...X[n], a permutation of 
{1,2,...,n}, by its inverse. This algorithm is due to Bing-Chao Huang [Inf. 
Proc. Letters 12 (1981), 237-238]. 

I1. [Initialize.] Set m 4+ n, j +} —1. 

I2. [Next element.] Set i «+ X[m]. Ifi < 0, go to step I5 (the element has 
already been processed). 

I3. [Invert one.] (At this point j < 0 and i = X[m]. If m is not the largest ele- 
ment of its cycle, the original permutation had X[—j] = m.) Set X[m] + j, 
jm, m + i, i + X[m]. 

I4. [End of cycle?] If i > 0, go back to I3 (the cycle has not ended); otherwise 
set i + j. (In the latter case, the original permutation had X[—j] = m, and 
m is largest in its cycle.) 


I5. [Store final value.] Set X[m] + —i. (Originally X|[—i] was equal to m.) 


I6. [Loop on m.] Decrease m by 1. If m > 0, go back to I2; otherwise the 
algorithm terminates. {Į 


See Table 3 for an example of this algorithm. The method is based on inversion 
of successive cycles of the permutation, tagging the inverted elements by making 
them negative, afterwards restoring the correct sign. 

Algorithm I resembles parts of Algorithm A, and it very strongly resembles 
the cycle-finding algorithm in Program B (lines 54-68). Thus it is typical 
of a number of algorithms involving rearrangements. When preparing a MIX 
implementation, we find that it is most convenient to keep the value of —i in a 
register instead of i itself: 
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Table 3 
COMPUTING THE INVERSE OF 621543 BY ALGORITHM I 


After step: I2 [3 13 IB I 12 B B I5 


= 
bo 
=m 
(Si 
= 
(Si 
= 
w 
= 
Ot 
— 
Ot 


X/1] 6 6 6 3 3 3 3 3 3 3 3 3 3 3 3 

X [2] 2 2 2 2 2 2 2 2 2 2 2 2 —4 2 2 

X [3] 1 1 6 6 6 6 6 6 6 6 6 6 6 6 6 

X (4) 5 5 5 5 5 5 5 —5 —5 —5 5 5 5 5 5 

X[5] 4 4 4 4 4 4 -1 -1 4 4 4 4 4 4 4 

X [6] 3 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 

m 6 3 1 6 6 5 4 5 5 4 4 3 2 2 1 

j 1 6 3 1 1 1 5 4 4 4 4 4 2 2 2 

i 3 1 6 —1 —1 4 5 1 4 5 5 6 4 2 3 

Read the columns from left to right. At point *, the cycle (163) has been inverted. 

Program I (Inverse in place). rll = m; r12 = —i; r13 = j; and n = N, a symbol 


to be defined when this program is assembled as part of a larger routine. 
01 INVERT ENT1 N 1 Il. Initialize. m <n. 


02 ENT3 -1 1 j&-1. 

03 2H LD2N X,1 N P. Next element. i+ X[m]. 

04 J2P 5F N Toldifi<0. 

05 3H ST3 X,1 N I3. Invert one. X[m] < j. 

06 ENN3 0,1 N js —m. 

07 ENN1 0,2 N mei. 

08 LD2N X,1 N i¢ X[n]. 

09 4H J2N 3B N I4. End of cycle? To 13 if i > 0. 
10 ENN2 0,3 C Otherwise set i + j. 

11 5H ST2 X,1 N I5. Store final value. X[m] < —i. 
12 6H DEC1 1 N I6. Loop on m. 

13 JiP 2B N Tol2ifm>O0. I 


The timing for this program is easily worked out in the manner shown earlier; 
every element X[m] is set first to a negative value in step I3 and later to a positive 
value in step I5. The total time comes to (14N + C + 2)u, where N is the size of 
the array and C is the total number of cycles. The behavior of C in a random 
permutation is analyzed below. 

There is almost always more than one algorithm to do any given task, so 
we would expect that there may be another way to invert a permutation. The 
following ingenious algorithm is due to J. Boothroyd: 


Algorithm J (Inverse in place). This algorithm has the same effect as Algo- 
rithm I but uses a different method. 
J1. [Negate all.] Set X[k] — —X [k], for 1 < k < n. Also set m + n. 


J2. [Initialize j.] Set j < m. 


I 

J3. [Find negative entry.] Set i + X[j]. If i > 0, set j 4 i and repeat this step. 
JA. [Invert.] Set X[j] < X[-7], X[—i] < m. 

J5. [Loop on m.] Decrease m by 1; if m > 0, go back to J2. Otherwise the 


algorithm terminates. J 
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Table 4 

COMPUTING THE INVERSE OF 621543 BY ALGORITHM J 
After step: J2 J3 J5 J3 J5 J3 J5 J3 J5 J3 J5 J3 J5 
XxX[1] 6 6 6 6 6 6 6 6 3 3 3 3 3 
X[2] 2 2 2 2 2 2 2 2 2 2 2 2 2 
X[8] -1 —1 6 6 6 6 6 6 6 6 6 6 6 
X[4]] -5 -5 -5 —5 5 5 5 5 5 5 5 5 5 
X[5] 4 4 4 4 5 5 4 4 4 4 4 4 4 
x[6] -3 -3 -1 -1 -1 -1 -1 -1 -6 -6 -6 -6 1 
m 6 6 5 5 4 4 3 3 2 2 1 1 0 
i 3 3 4 4 5 5 1 1 2 2 6 6 
j 6 6 6 5 5 5 5 6 6 2 2 6 6 


See Table 4 for an example of Boothroyd’s algorithm. Again the method is 
essentially based on the cycle structure, but this time it is less obvious that the 
algorithm really works! Verification is left to the reader (see exercise 13). 


Program J (Analogous to Program I). rll = m; rI2 = j; r13 = —i. 


01 INVERT ENN1 N 1 Jl. Negate all. 

02 ST1 X+N+1,1(0:0) WN Set sign negative. 

03 INC1 1 N 

04 JiN *-2 N More? 

05 ENT1 N 1 men. 

06 2H ENN3 0,1 N J2. Initialize j. im. 
07 ENN2 0,3 A jsi 

08 LD3N X,2 A J3. Find negative entry. 
09 J3N *-2 A i>0? 

10 LDA X,3 N J4. Invert. 

11 STA X,2 N X[j] + X[-i]. 

12 STi X,3 N X|-il m. 

13 DEC1 1 N J5. Loop on m. 

14 JiP 2B N ToJ2ifm>0. I 


To decide how fast this program runs, we need to know the quantity A; this 
quantity is so interesting and instructive, it has been left as an exercise (see 
exercise 14). 

Although Algorithm J is deucedly clever, analysis shows that Algorithm I is 
definitely superior. In fact, the average running time of Algorithm J turns out 
to be essentially proportional to nlnn, while that of Algorithm I is essentially 
proportional to n. Maybe some day someone will find a use for Algorithm J (or 
some related modification); it is a bit too pretty to be forgotten altogether. 


An unusual correspondence. We have already remarked that the cycle 
notation for a permutation is not unique; the six-element permutation (1 6 3)(4 5) 
may be written (5 4)(3 1 6), etc. It will be useful to consider a canonical form 
for the cyclic notation; the canonical form is unique. To get the canonical form, 
proceed as follows: 


a) Write all singleton cycles explicitly. 
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b) Within each cycle, put the smallest number first. 
c) Order the cycles in decreasing order of the first number in the cycle. 


For example, starting with (3 1 6)(5 4) we would get 
(a): (3 1.6) (5 4) (2); (b): (16 3) (45) (2); (c): (45) (2) (1 6 3). (20) 


The important property of this canonical form is that the parentheses may 
be dropped and uniquely reconstructed again. Thus there is only one way to 
insert parentheses in “4 5 2 1 6 3” to get a canonical cycle form: One must insert 
a left parenthesis just before each left-to-right minimum (namely, just before 
each element that is preceded by no smaller elements). 

This insertion and removal of parentheses gives us an unusual one-to-one 
correspondence between the set of all permutations expressed in cycle form 
and the set of all permutations expressed in linear form. For example, the 
permutation 6 2 1 5 4 3 in canonical cycle form is (4 5) (2) (1 6 3); remove 
parentheses to get 4 5 2 1 6 3, which in cycle form is (2 5 6 3) (1 4); remove 
parentheses to get 2 5 6 3 1 4, which in cycle form is (3 6 4) (1 2 5); etc. 

This correspondence has numerous applications to the study of permutations 
of different types. For example, let us ask “How many cycles does a permutation 
of n elements have, on the average?” To answer this question we consider the set 
of all n! permutations expressed in canonical form, and drop the parentheses; we 
are left with the set of all n! permutations in some order. Our original question 
is therefore equivalent to, “How many left-to-right minima does a permutation of 
n elements have, on the average?” We have already answered the latter question 
in Section 1.2.10; this was the quantity (A+ 1) in the analysis of Algorithm 
1.2.10M, for which we found the statistics 


min 1, ave Hp, maxn, dev ,/H,— AH, (21) 


(Actually, we discussed the average number of right-to-left maxima, but that’s 
clearly the same as the number of left-to-right minima.) Furthermore, we proved 
in essence that a permutation of n objects has k left-to-right minima with prob- 
ability 2] /n!; therefore a permutation of n objects has k cycles with probability 
] /nt. 

We can also ask about the average distance between left-to-right minima, 
which becomes equivalent to the average length of a cycle. By (21), the total 
number of cycles among all the n! permutations is n! Hn, since it is n! times the 
average number of cycles. If we pick one of these cycles at random, what is its 
average length? 

Imagine all n! permutations of {1,2,...,n} written down in cycle notation; 
how many three-cycles are present? To answer this question, let us consider how 
many times a particular three-cycle (x y z) appears: It clearly appears in exactly 
(n — 3)! of the permutations, since this is the number of ways the remaining 
n — 3 elements may be permuted. Now the number of different possible three- 
cycles (xyz) is n(n — 1)(n — 2)/8, since there are n choices for x, (n — 1) for y, 
(n — 2) for z, and among these n(n — 1)(n — 2) choices each different three-cycle 
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has appeared in three forms (xyz), (yz x), (zxy). Therefore the total number 
of three-cycles among all n! permutations is n(n — 1)(n — 2)/3 times (n — 3)!, 
namely n!/3. Similarly, the total number of m-cycles is n!/m, for 1 < m <n. 
(This provides another simple proof of the fact that the total number of cycles 
is n! Hn; hence the average number of cycles in a random permutation is Hy, 
as we already knew.) Exercise 17 shows that the average length of a randomly 
chosen cycle is n/H,, if we consider the n! H, cycles to be equally probable; but 
if we choose an element at random in a random permutation, the average length 
of the cycle containing that element is somewhat greater than n/Hn. 

To complete our analyses of Algorithms A and B, we would like to know 
the average number of singleton cycles in a random permutation. This is an 
interesting problem. Suppose we write down the n! permutations, listing first 
those with no singleton cycles, then those with just one, etc.; for example, if 
n=A, 

no fixed elements: 2143 2341 2413 3142 3412 3421 4123 4312 4321 

one fixed element: 1342 1423 3241 4213 2431 4132 2314 3124 

two fixed elements: 1243 1432 1324 4231 3214 2134 
three fixed elements: 


four fixed elements: 1234 


(Singleton cycles, which are the elements that remain fixed by a permutation, 
have been specially marked in this list.) Permutations with no fixed elements 
are called derangements; the number of derangements is the number of ways to 
put n letters into n envelopes, getting them all wrong. 

Let Pag be the number of permutations of n objects having exactly k fixed 
elements, so that for example, 


Po =9, Pa =8, Pye =6, Pag =0, Pag =l. 


An examination of the list above reveals the principal relationship between these 
numbers: We can get all permutations with k fixed elements by first choosing 
the k that are to be fixed (this can be done in (7) ways) and then permuting 
the remaining n — k elements in all P(,_,)o ways that leave no further elements 


fixed. Hence 
n 


Pak = (2) Papa: (22) 
We also have the rule that “the whole is the sum of its parts”: 
n! = Pan + Paine + Pain-2) tay He (23) 
Combining Eqs. (22) and (23) and rewriting the result slightly, we find that 


Poo Pio P30 
n 

0! 1! 2! 3! 

an equation that must be true for all positive integers n. This equation has 

already confronted us before—it appears in Section 1.2.5 in connection with 


n! = 


t n(n — 1) + n(n — 1)(n — 2) forse, (24) 
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Stirling’s attempt to generalize the factorial function—and we found a simple 
derivation of its coefficients in Section 1.2.6 (Example 5). We conclude that 
Pino 1 1 1 
= (25) 


m oe 
Now let pnk be the probability that a permutation of n objects has exactly 
k singleton cycles. Since Png = Pak/n!, we have from Eqs. (22) and (25) 


1 1 o1 a al 
meS (: Ta | ee aa) 2) 


The generating function G,,(z) = Pno + Pniz + Pn22? +- is therefore 
1 e 1 i 
Gaz) =1+ -Dte e-s E ge- (e) 
| n! ~ j! 
O<j<n 


From this formula it follows that G! (z) = Gn-1(z), and with the methods of 
Section 1.2.10 we obtain the following statistics on the number of singleton cycles: 


(min 0, avel, max n, dev 1), if n>2. (28) 


A somewhat more direct way to count the number of permutations having 
no singleton cycles follows from the principle of inclusion and exclusion, which is 
an important method for many enumeration problems. The general principle of 
inclusion and exclusion may be formulated as follows: We are given N elements, 
and M subsets, S1, S2,..., Sm, of these elements; and our goal is to count how 
many of the elements lie in none of the subsets. Let |S] denote the number of 
elements in a set S; then the desired number of objects in none of the sets 5S; is 


N- >> |Sjl+ X IS;NSI- SS [S:N5;05,|+--- 


1<j<M 1<j<k<M 1<i<jg<k<M 
+ (—1)¥ [Sy M.A Sm| (29) 


(Thus we first subtract the number of elements in S1,..., Sm from the total 
number, N; but this underestimates the desired total. So we add back the 
number of elements that are common to pairs of sets, Sj N Sk, for each pair 
Sj and S;; this, however, gives an overestimate. So we subtract the elements 
common to triples of sets, etc.) There are several ways to prove this formula, 
and the reader is invited to discover one of them. (See exercise 25.) 

To count the number of permutations on n elements having no singleton 
cycles, we consider the N = n! permutations and let S} be the set of permutations 
in which element j forms a singleton cycle. If 1 < jı < jo < +--+ < jk < n, the 
number of elements in Sj, N Sja N +++ N Sj, is the number of permutations in 
which j1,...,j% are singleton cycles, and this is clearly (n — k)!. Thus formula 
(29) becomes 


gi (") m=i (3) (n—2)!— (z) (n3)! +--+ (-1)” (") 0!, 


in agreement with (25). 


v v 
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The principle of inclusion and exclusion is due to A. de Moivre [see his 
Doctrine of Chances (London: 1718), 61-63; 3rd ed. (1756, reprinted by Chelsea, 
1957), 110-112], but its significance was not generally appreciated until it was 
popularized and developed further by I. Todhunter in his Algebra (second edi- 
tion, 1860), §762, and by W. A. Whitworth in the well-known book Choice and 
Chance (Cambridge: 1867). 

Combinatorial properties of permutations are explored further in Section 5.1. 


EXERCISES 

1. [02] Consider the transformation of {0, 1, 2,3,4,5,6} that replaces x by 2x mod 7. 
Show that this transformation is a permutation, and write it in cycle form. 

2. [10] The text shows how we might set (a,b,c, d,e, f) < (c,d, f,b,e,a) by using a 
series of replacement operations (x + y) and one auxiliary variable t. Show how to do 


the job by using a series of exchange operations (x + y) and no auxiliary variables. 


3. [03] Compute the product (% 5 ¢ ¢ $ txe? 5 Ter 


), and express the answer 
in two-line notation. (Compare with Eq. (4).) 

4. [10] Express (abd)(e f)(ac f)(bd) as a product of disjoint cycles. 

5. [M10] Equation (3) shows several equivalent ways to express the same permuta- 
tion in cycle form. How many different ways of writing that permutation are possible, 


if all singleton cycles are suppressed? 


6. [M28] What changes are made to the timing of Program A if we remove the 
assumption that all blank words occur at the extreme right? 

7. [10] IfProgram A is presented with the input (6), what are the quantities X, Y, M, 
N, U, and V of (19)? What is the time required by Program A, excluding input-output? 

8. [23] Would it be feasible to modify Algorithm B to go from left to right instead 
of from right to left through the input? 

9. [10] Both Programs A and B accept the same input and give the answer in 
essentially the same form. Is the output exactly the same under both programs? 


10. [M28] Examine the timing characteristics of Program B, namely, the quantities 
A, B, ..., Z shown there; express the total time in terms of the quantities X, Y, M, N, 
U, V defined in (19), and of F. Compare the total time for Program B with the total 
time for Program A on the input (6), as computed in exercise 7. 

11. [15] Find a simple rule for writing 7~ in cycle form, if the permutation 7 is given 
in cycle form. 

12. [M27] (Transposing a rectangular matrix.) Suppose an m xn matrix (aij), Mm Æ n, 
is stored in memory in a fashion like that of exercise 1.3.2-10, so that the value of aij 
appears in location L+n(i—1)+(j—1), where L is the location of a11. The problem is 
to find a way to transpose this matrix, obtaining an n x m matrix (bij), where bij = aji 
is stored in location L + m(i — 1) + (j — 1). Thus the matrix is to be transposed “on 
itself.” (a) Show that the transposition transformation moves the value that appears 
in cell L + x to cell L + (ma mod N), for all x in the range 0 < x < N = mn-1. 
(b) Discuss methods for doing this transposition by computer. 


13. [M24] Prove that Algorithm J is valid. 
14. [M34] Find the average value of the quantity A in the timing of Algorithm J. 


15. [M12] Is there a permutation that represents exactly the same transformation 
both in the canonical cycle form without parentheses and in the linear form? 
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16. [M15] Start with the permutation 1324 in linear notation; convert it to canonical 
cycle form and then remove the parentheses; repeat this process until arriving at the 
original permutation. What permutations occur during this process? 


17. [M24] (a) The text demonstrates that there are n! Hn cycles altogether, among 
all the permutations on n elements. If these cycles (including singleton cycles) are 
individually written on n! H,, slips of paper, and if one of these slips of paper is chosen 
at random, what is the average length of the cycle that is thereby picked? (b) If we write 
the n! permutations on n! slips of paper, and if we choose a number k at random and 
also choose one of the slips of paper, what is the probability that the cycle containing 
k on that slip is an m-cycle? What is the average length of the cycle containing k? 


18. [M27] What is prem, the probability that a permutation of n objects has exactly 
k cycles of length m? What is the corresponding generating function Gnm(z)? What 
is the average number of m-cycles and what is the standard deviation? (The text 
considers only the case m = 1.) 

19. [HM21] Show that, in the notation of Eq. (25), the number Pro of derangements 
is exactly equal to n!/e rounded to the nearest integer, for all n > 1. 

20. [M20] Given that all singleton cycles are written out explicitly, how many different 
ways are there to write the cycle notation of a permutation that has a1 one-cycles, a2 
two-cycles, ...? (See exercise 5.) 

21. [M22] What is the probability P(n;a1,a2,...) that a permutation of n objects 
has exactly ai one-cycles, a2 two-cycles, etc.? 

22. [HM34] (The following approach, due to L. Shepp and S. P. Lloyd, gives a con- 
venient and powerful method for solving problems related to the cycle structure of 


random permutations.) Instead of regarding the number, n, of objects as fixed, and 
the permutation variable, let us assume instead that we independently choose the 


quantities a1, @2, @3,... appearing in exercises 20 and 21 according to some probability 
distribution. Let w be any real number between 0 and 1. 
a) Suppose that we choose the random variables a1, a2, a3, ... according to the rule 


that “the probability that am = k is f(w,m,k),” for some function f(w,m, k). 
Determine the value of f(w,m,k) so that the following two conditions hold: 
(i) Vaso f(w,m,k) = 1, for 0 < w < 1 and m > 1; (ii) the probability that 


ay + 2a2 + 3&3 + --- = n and that ai = ki, ag = k2, a3 = kz, ... equals 
(1 — w)w” P(n; kı, k2, k3,...), where P(n; kı, k2,k3,...) is defined in exercise 21. 
b) A permutation whose cycle structure is aj, a2, a3, ... clearly permutes exactly 
ai1+2a2+3a3+--- objects. Show that if the a’s are randomly chosen according to 
the probability distribution in part (a), the probability that a1 +2a2+3a3+:--=n 
is (1 — w)w”; the probability that a, + 2a2 + 3a3+4+--- is infinite is zero. 
c) Let ¢(a1,a2,...) be any function of the infinitely many numbers ay, a2, .... 


Show that if the a’s are chosen according to the probability distribution in (a), 
the average value of ġ is (1 — w) 0,5, W” gn; here ¢n denotes the average value 
of ¢ taken over all permutations of n objects, where the variable a; represents 
the number of j-cycles of a permutation. [For example, if d(a1,a2,...) = a1, the 
value of n is the average number of singleton cycles in a random permutation of 
n objects; we showed in (28) that n = 1 for all n.] 

d) Use this method to find the average number of cycles of even length in a random 
permutation of n objects. 

e) Use this method to solve exercise 18. 
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23. [HM42] (Golomb, Shepp, Lloyd.) If ln denotes the average length of the longest 
cycle in a permutation of n objects, show that ln ~ An + 3A, where A & 0.62433 is a 
constant. Prove in fact that limn—oo(In — An — 5A) = 0. 


24. [M41] Find the variance of the quantity A that enters into the timing of Algo- 
rithm J. (See exercise 14.) 


25. [M22] Prove Eq. (29). 


> 26. [M24] Extend the principle of inclusion and exclusion to obtain a formula for 
the number of elements that are in exactly r of the subsets $1, S2,..., Sm. (The text 
considers only the case r = 0.) 


27. [M20] Use the principle of inclusion and exclusion to count the number of integers 
n in the range 0 < n < amimz2...m: that are not divisible by any of m1,ma,..., Mt. 
Here m1, m2, ..., Mt, and a are positive integers, with mj L Mk when j # k. 


28. [M21] (I. Kaplansky.) If the “Josephus permutation” defined in exercise 1.3.2- 
22 is expressed in cycle form, we obtain (1 5 3 6 8 2 4)(7) when n = 8 and m = 4. 
Show that this permutation in the general case is the product (n n—1 ... 2.1)'~' x 
(nn—-1... 2)™1...(nn—-1)™1. 

29. [M25] Prove that the cycle form of the Josephus permutation when m = 2 can be 
obtained by first expressing the “perfect shuffle” permutation of {1,2,...,2n}, which 
takes (1,2,...,2n) into (2,4,...,2n,1,3,...,2n—1), in cycle form, then reversing left 
and right and erasing all the numbers greater than n. For example, when n = 11 the 
perfect shuffle is (1 2 4 8 16 9 18 13 3 6 12)(5 10 20 17 11 22 21 19 15 7 14) and the 
Josephus permutation is (7 11 10 5)(6 3 9 8 4 2 1). 


30. [M24] Use exercise 29 to show that the fixed elements of the Josephus permutation 
when m = 2 are precisely the numbers (2% — 1)(2n + 1)/(2°*! — 1) for all positive 
integers d such that this is an integer. 


31. [HM38]| Generalizing exercises 29 and 30, prove that the jth man to be executed, 
for general m and n, is in position x, where x may be computed as follows: Set x «+ jm; 
then, while x > n, set x + |(m(x — n) — 1)/(m — 1)|. Consequently the average 
number of fixed elements, for 1 < n < N and fixed m > 1 as N —> on, approaches 
Ys 1(m-— 1)*/(m**" — (m—1)*). [Since this value lies between (m — 1)/m and 1, the 
Josephus permutations have slightly fewer fixed elements than random ones do.] 


32. [M25] (a) Prove that any permutation 7 = m172...72m+1 of the form 
T = (23)°?(4.5)°4... (2m 2m-+1)2" (1 2) (3 4)... (2m—1 2m) 2-4, 


where each ex is 0 or 1, has |m, — k| < 2 for 1 < k < 2m +1. 
(b) Given any permutation p of {1,2,...,n}, construct a permutation 7 of the 
stated form such that pr is a single cycle. Thus every permutation is “near” a cycle. 


33. [M33] Ifm= 2?” andn = 27!+1 show how to construct sequences of permutations 
(&j1, Qj2, -< , Qjn; Bil, Bj2,---,Bjn) for 0 < j < m with the following “orthogonality” 
property: 


12345), ifi=j; 
041 bj1Qi2bj2 . . - Qinbjn = { . ) ee 


Each aj, and jk should be a permutation of {1, 2,3, 4,5}. 


> 34. [M25] (Transposing blocks of data.) One of the most common permutations 
needed in practice is the change from af to Ba, where a and £ are substrings of an array. 


1.3.3 APPLICATIONS TO PERMUTATIONS 185 


In other words, if vo%1...%m—1 = @ and YmXm41---Lm4+n—1 = , we want to change 
the array %0%1...%m+n—-1 = Q to the array tmUm+1-..-Lm+n—-1L0L1..-Lm-1 = Ba; 
each element x, should be replaced by £p) for 0 < k < m+n, where p(k) = 
(k +m) mod (m+n). Show that every such “cyclic-shift” permutation has a simple 
cycle structure, and exploit that structure to devise a simple algorithm for the desired 
rearrangement. 


35. [M30] Continuing the previous exercise, let ro%1...2i+m4n—-1 = GY where a, 
B, and y are strings of respective lengths l, m, and n, and suppose that we want to 
change aby to yBa. Show that the corresponding permutation has a convenient cycle 
structure that leads to an efficient algorithm. [Exercise 34 considered the special case 
m = 0.] Hint: Consider changing (a@f)(y8) to (78)(aB). 


36. [27] Write a MIX subroutine for the algorithm in the answer to exercise 35, and 
analyze its running time. Compare it with the simpler method that goes from ay to 
(aBy)" = 7" B" a" to yBa, where o” denotes the left-right reversal of the string o. 


37. [M26] (Even permutations.) Let m be a permutation of {1,...,n}. Prove that m 
can be written as the product of an even number of 2-cycles if and only if 7 can be 
written as the product of exactly two n-cycles. 
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1.4. SOME FUNDAMENTAL PROGRAMMING TECHNIQUES 
1.4.1. Subroutines 


WHEN A CERTAIN task is to be performed at several different places in a program, 
it is usually undesirable to repeat the coding in each place. To avoid this 
situation, the coding (called a subroutine) can be put into one place only, and a 
few extra instructions can be added to restart the outer program properly after 
the subroutine is finished. Transfer of control between subroutines and main 
programs is called subroutine linkage. 

Each machine has its own peculiar manner for achieving efficient subroutine 
linkage, usually involving special instructions. In MIX, the J-register is used for 
this purpose; our discussion will be based on MIX machine language, but similar 
remarks will apply to subroutine linkage on other computers. 

Subroutines are used to save space in a program; they do not save any time, 
other than the time implicitly saved by occupying less space — for example, less 
time to load the program, or fewer passes necessary in the program, or better use 
of high-speed memory on machines with several grades of memory. The extra 
time taken to enter and leave a subroutine is usually negligible. 

Subroutines have several other advantages. They make it easier to visualize 
the structure of a large and complex program; they form a logical segmentation 
of the entire problem, and this usually makes debugging of the program easier. 
Many subroutines have additional value because they can be used by people 
other than the programmer of the subroutine. 

Most computer installations have built up a large library of useful sub- 
routines, and such a library greatly facilitates the programming of standard 
computer applications that arise. A programmer should not think of this as the 
only purpose of subroutines, however; subroutines should not always be regarded 
as general-purpose programs to be used by the community. Special-purpose 
subroutines are just as important, even when they are intended to appear in 
only one program. Section 1.4.3.1 contains several typical examples. 

The simplest subroutines are those that have only one entrance and one exit, 
such as the MAXIMUM subroutine we have already considered (see Section 1.3.2, 
Program M). For reference, we will recopy that program here, changing it so 
that a fixed number of cells, 100, is searched for the maximum: 

* MAXIMUM OF X[1..100] 


MAX100 STJ EXIT Subroutine linkage 
ENT3 100 M1. Initialize. 


JMP 2F 
1H CMPA X,3 M3. Compare. 

JGE *+3 (1) 
2H ENT2 0,3 M4. Change m. 

LDA X,3 New maximum found 

DEC3 1 M5. Decrease k. 


J3P 1B M2. All tested? 
EXIT JMP * Return to main program. J] 
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In a larger program containing this coding as a subroutine, the single instruction 
‘JMP MAX100’ would cause register A to be set to the current maximum value 
of locations X + 1 through X + 100, and the position of the maximum would 
appear in rl2. Subroutine linkage in this case is achieved by the instructions 
‘MAX100 STJ EXIT’ and, later, ‘EXIT JMP *’. Because of the way the J-register 
operates, the exit instruction will then jump to the location following the place 
where the original reference to MAX100 was made. 


as, Newer computers, such as the machine MMIX that is destined to replace MIX, 

{£ have better ways to remember return addresses. The main difference is that 
program instructions are no longer modified in memory; the relevant information 
is kept in registers or in a special array, not within the program itself. (See 
exercise 7.) The next edition of this book will adopt the modern view, but for 
now we will stick to the old-time practice of self-modifying code. 


It is not hard to obtain quantitative statements about the amount of code 
saved and the amount of time lost when subroutines are used. Suppose that 
a piece of coding requires k locations and that it appears in m places in the 
program. Rewriting this as a subroutine, we need an extra instruction STJ and 
an exit line for the subroutine, plus a single JMP instruction in each of the m 
places where the subroutine is called. This gives a total of m + k + 2 locations, 
rather than mk, so the amount saved is 


(m—1)(k-1)-3. (2) 


If k is 1 or m is 1 we cannot possibly save any space by using subroutines; this, 
of course, is obvious. If k is 2, m must be greater than 4 in order to gain, etc. 

The amount of time lost is the time taken for the extra JMP, STJ, and JMP 
instructions, which are not present if the subroutine is not used; therefore if the 
subroutine is used ¢ times during a run of the program, 4t extra cycles of time 
are required. 

These estimates must be taken with a grain of salt, because they were given 
for an idealized situation. Many subroutines cannot be called simply with a 
single JMP instruction. Furthermore, if the coding is repeated in many parts of a 
program, without using a subroutine approach, the coding for each part can be 
customized to take advantage of special characteristics of the particular part of 
the program in which it lies. With a subroutine, on the other hand, the coding 
must be written for the most general case, not a specific case, and this will often 
add several additional instructions. 


When a subroutine is written to handle a general case, it is expressed in 
terms of parameters. Parameters are values that govern the subroutine’s actions; 
they are subject to change from one call of the subroutine to another. 

The coding in the outside program that transfers control to the subroutine 
and gets it properly started is known as the calling sequence. Particular values 
of parameters, supplied when the subroutine is called, are known as arguments. 
With our MAX100 subroutine, the calling sequence is simply ‘JMP MAX100’, but 
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a longer calling sequence is generally necessary when arguments must be sup- 
plied. For example, Program 1.3.2M is a generalization of MAX100 that finds the 
maximum of the first n elements of the table. The parameter n appears in index 
register 1, and its calling sequence 

LD1 =n= ENT n 

JMP MAXIMUM JMP MAXIMUM 


involves two steps. 
If the calling sequence takes c memory locations, formula (2) for the amount 
of space saved changes to 


(m — 1) (k — c) — constant (3) 


and the time lost for subroutine linkage is slightly increased. 

A further correction to the formulas above can be necessary because certain 
registers might need to be saved and restored. For example, in the MAX100 
subroutine, we must remember that by writing ‘JMP MAX100’ we are not only 
getting the maximum value in register A and its position in register I2; we are 
also setting register I3 to zero. A subroutine may destroy register contents, and 
this must be kept in mind. In order to prevent MAX100 from changing the setting 
of rI3, it would be necessary to include additional instructions. The shortest and 
fastest way to do this with MIX would be to insert the instruction ‘ST3 3F (0:2) 
just after MAX100 and then ‘3H ENT3 *’ just before EXIT. The net cost would 
be an extra two lines of code, plus three machine cycles on every call of the 
subroutine. 

A subroutine may be regarded as an extension of the computer’s machine 
language. With the MAX100 subroutine in memory, we now have a single instruc- 
tion (namely, ‘JMP MAX100’) that is a maximum-finder. It is important to define 
the effect of each subroutine just as carefully as the machine language operators 
themselves have been defined; a programmer should therefore be sure to write 
down the characteristics of each subroutine, even though nobody else will be 
making use of the routine or its specification. In the case of MAXIMUM as given in 
Section 1.3.2, the characteristics are as follows: 


Calling sequence: JMP MAXIMUM. 

Entry conditions: rll = n; assume that n > 1. 

Exit conditions: rA = max CONTENTS (X + k) = CONTENTS (x + r12); (4) 
SESN 


rI3 = 0; rJ and CI are also affected. 


(We will customarily omit mention of the fact that register J and the compar- 
ison indicator are affected by a subroutine; it has been mentioned here only 
for completeness.) Note that rX and rIl are unaffected by the action of the 
subroutine, for otherwise these registers would have been mentioned in the exit 
conditions. A specification should also mention all memory locations external 
to the subroutine that might be affected; in this case the specification allows us 
to conclude that nothing has been stored, since (4) doesn’t say anything about 
changes to memory. 


1.4.1 SUBROUTINES 189 


Now let’s consider multiple entrances to subroutines. Suppose we have a 
program that requires the general subroutine MAXIMUM, but it usually wants to use 
the special case MAX100 in which n = 100. The two can be combined as follows: 


MAX100 ENT3 100 First entrance 
MAXN STJ EXIT Second entrance 
JMP 2F Continue as in (1). (5) 


EXIT JMP * Return to main program. J 


Subroutine (5) is essentially the same as (1), with the first two instructions 
interchanged; we have used the fact that ‘ENT3’ does not change the setting of 
the J-register. If we wanted to add a third entrance, MAX50, to this subroutine, 
we could insert the code 


MAX50 ENT3 50 (6) 
JSJ MAXN 


at the beginning. (Recall that ‘JSJ’ means jump without changing register J.) 

When the number of parameters is small, it is often desirable to transmit 
them to a subroutine either by having them in convenient registers (as we have 
used rI3 to hold the parameter n in MAXN and as we used rI1 to hold the parameter 
n in MAXIMUM), or by storing them in fixed memory cells. 

Another convenient way to supply arguments is simply to list them after 
the JMP instruction; the subroutine can refer to its parameters because it knows 
the J-register setting. For example, if we wanted to make the calling sequence 
for MAXN be 

JMP MAXN 
CON n (7) 


then the subroutine could be written as follows: 
MAXN STJ *+1 
ENT1 * rll rJ. 


LD3 0,1 ril3<¢n. 
JMP 2F Continue as in (1). (8) 


J3P 1B 
JMP 1,1 Return. J 


On machines like System /360, for which linkage is ordinarily done by putting the 
exit location in an index register, a convention like this is particularly convenient. 
It is also useful when a subroutine needs many arguments, or when a program 
has been written by a compiler. The technique of multiple entrances that we 
used above often fails in this case, however. We could “fake it” by writing 


MAX100 STJ 1F 


JMP MAXN 
CON 100 
1H JMP * l 


but this is not as attractive as (5). 
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A technique similar to that of listing arguments after the jump is normally 
used for subroutines with multiple exits. Multiple exit means that we want the 
subroutine to return to one of several different locations, depending on conditions 
detected by the subroutine. In the strictest sense, the location to which a 
subroutine exits is a parameter; so if there are several places to which it might 
exit, depending on the circumstances, they should be supplied as arguments. 
Our final example of the “maximum” subroutine will have two entrances and 
two exits. The calling sequence is: 


For general n For n = 100 
ENT3 n 
JMP MAXN JMP MAX100 
Exit here if max < 0 or max > rX. Exit here if max < 0 or max > rX. 
Exit here if 0 < max < rX. Exit here if 0 < max < rX. 


(In other words, exit is made to the location two past the jump when the 
maximum value is positive and less than the contents of register X.) The 
subroutine for these conditions is easily written: 


MAX100 ENT3 100 Entrance for n = 100 


MAXN STJ EXIT Entrance for general n 
JMP 2F Continue as in (1). 


J3P 1B 

JANP EXIT Take normal exit if the max is < 0. (9) 
STX TEMP 

CMPA TEMP 

JGE EXIT ‘Take normal exit if the max is > rX. 

ENT3 1 Otherwise take the second exit. 


EXIT JMP *,3 Return to proper place. Jj 


Subroutines may call on other subroutines; in complicated programs it is not 
unusual to have subroutine calls nested more than five deep. The only restriction 
that must be followed when using linkage as described here is that no subroutine 
may call on any other subroutine that is (directly or indirectly) calling on it. For 
example, consider the following scenario: 


[Main program] [Subroutine A] [Subroutine B] [Subroutine C] 
A STJ EXITA | B STJ EXITB | C STJ EXITC 
JMP A JMP B JMP C JMP A 
EXITA JMP * EXITB JMP * EXITC JMP * (10) 


If the main program calls on A, which calls B, which calls C, and then C 
calls on A, the address in EXITA referring to the main program is destroyed, 
and there is no way to return to that program. A similar remark applies to all 
temporary storage cells and registers used by each subroutine. It is not difficult to 
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devise subroutine linkage conventions that will handle such recursive situations 
properly; Chapter 8 considers recursion in detail. 


We conclude this section by discussing briefly how we might go about writing 
a complex and lengthy program. How can we decide what kind of subroutines 
we will need, and what calling sequences should be used? One successful way to 
determine this is to use an iterative procedure: 


Step 0 (Initial idea). First we decide vaguely upon the general plan of 
attack that the program will use. 

Step 1 (A rough sketch of the program). We start now by writing the “outer 
levels” of the program, in any convenient language. A somewhat systematic way 
to go about this has been described very nicely by E. W. Dijkstra, Structured 
Programming (Academic Press, 1972), Chapter 1, and by N. Wirth, CACM 
14 (1971), 221-227. We may begin by breaking the whole program into a 
small number of pieces, which might be thought of temporarily as subroutines, 
although they are called only once. These pieces are successively refined into 
smaller and smaller parts, having correspondingly simpler jobs to do. Whenever 
some computational task arises that seems likely to occur elsewhere or that has 
already occurred elsewhere, we define a subroutine (a real one) to do that job. 
We do not write the subroutine at this point; we continue writing the main 
program, assuming that the subroutine has performed its task. Finally, when 
the main program has been sketched, we tackle the subroutines in turn, trying 
to take the most complex subroutines first and then their sub-subroutines, etc. 
In this manner we will come up with a list of subroutines. The actual function 
of each subroutine has probably already changed several times, so that the first 
parts of our sketch will by now be incorrect; but that is no problem, it is merely 
a sketch. For each subroutine we now have a reasonably good idea about how it 
will be called and how general-purpose it should be. It usually pays to extend 
the generality of each subroutine a little. 

Step 2 (First working program). This step goes in the opposite direction 
from step 1. We now write in computer language, say MIXAL or PL/MIX or 
a higher-level language; we start this time with the lowest level subroutines, 
and do the main program last. As far as possible, we try never to write any 
instructions that call a subroutine before the subroutine itself has been coded. 
(In step 1, we tried the opposite, never considering a subroutine until all of its 
calls had been written.) 

As more and more subroutines are written during this process, our con- 
fidence gradually grows, since we are continually extending the power of the 
machine we are programming. After an individual subroutine is coded, we should 
immediately prepare a complete description of what it does, and what its calling 
sequences are, as in (4). It is also important not to overlay temporary storage 
cells; it may very well be disastrous if every subroutine refers to location TEMP, 
although when preparing the sketch in step 1, it was convenient not to worry 
about such problems. An obvious way to overcome overlay worries is to have 
each subroutine use only its own temporary storage, but if this is too wasteful 


192 BASIC CONCEPTS 1.4.1 


of space, another scheme that does fairly well is to name the cells TEMP1, TEMP2, 
etc.; the numbering within a subroutine starts with TEMP7, where j is one higher 
than the greatest number used by any of the sub-subroutines of this subroutine. 

Step 3 (Reexamination). The result of step 2 should be very nearly a 
working program, but it may be possible to improve on it. A good way is to 
reverse direction again, studying for each subroutine all of the calls made on it. 
It may well be that the subroutine should be enlarged to do some of the more 
common things that are always done by the outside routine just before or after 
it uses the subroutine. Perhaps several subroutines should be merged into one; 
or perhaps a subroutine is called only once and should not be a subroutine at 
all. (Perhaps a subroutine is never called and can be dispensed with entirely.) 

At this point, it is often a good idea to scrap everything and start over 
again at step 1! This is not intended to be a facetious remark; the time spent in 
getting this far has not been wasted, for we have learned a great deal about the 
problem. With hindsight, we will probably have discovered several improvements 
that could be made to the program’s overall organization. There’s no reason to 
be afraid to go back to step 1—it will be much easier to go through steps 2 
and 3 again, now that a similar program has been done already. Moreover, we 
will quite probably save as much debugging time later on as it will take to rewrite 
everything. Some of the best computer programs ever written owe much of their 
success to the fact that all the work was unintentionally lost, at about this stage, 
and the authors had to begin again. 

On the other hand, there is probably never a point when a complex computer 
program cannot be improved somehow, so steps 1 and 2 should not be repeated 
indefinitely. When significant improvement can clearly be made, it is well worth 
the additional time required to start over, but eventually a point of diminishing 
returns is reached. 

Step 4 (Debugging). After a final polishing of the program, including 
perhaps the allocation of storage and other last-minute details, it is time to 
look at it in still another direction from the three that were used in steps 1, 
2, and 3—now we study the program in the order in which the computer will 
perform it. This may be done by hand or, of course, by machine. The author 
has found it quite helpful at this point to make use of system routines that trace 
each instruction the first two times it is executed; it is important to rethink the 
ideas underlying the program and to check that everything is actually taking 
place as expected. 

Debugging is an art that needs much further study, and the way to approach 
it is highly dependent on the facilities available at each computer installation. 
A good start towards effective debugging is often the preparation of appropriate 
test data. The most effective debugging techniques seem to be those that are 
designed and built into the program itself—many of today’s best programmers 
will devote nearly half of their programs to facilitating the debugging process 
in the other half; the first half, which usually consists of fairly straightforward 
routines that display relevant information in a readable format, will eventually 
be thrown away, but the net result is a surprising gain in productivity. 
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Another good debugging practice is to keep a record of every mistake made. 
Even though this will probably be quite embarrassing, such information is in- 
valuable to anyone doing research on the debugging problem, and it will also 
help you learn how to reduce the number of future errors. 

Note: The author wrote most of the preceding comments in 1964, after he 
had successfully completed several medium-sized software projects but before 
he had developed a mature programming style. Later, during the 1980s, he 
learned that an additional technique, called structured documentation or literate 
programming, is probably even more important. A summary of his current beliefs 
about the best way to write programs of all kinds appears in the book Literate 
Programming (Cambridge Univ. Press, first published in 1992). Incidentally, 
Chapter 11 of that book contains a detailed record of all bugs removed from the 
TEX program during the period 1978-1991. 


Up to a point it is better to let the snags [bugs] be there 
than to spend such time in design that there are none 
(how many decades would this course take?). 


— A. M. TURING, Proposals for ACE (1945) 


EXERCISES 


1. [10] State the characteristics of subroutine (5), just as (4) gives the characteristics 
of Subroutine 1.3.2M. 


2. [10] Suggest code to substitute for (6) without using the JSJ instruction. 


3. [M15] Complete the information in (4) by stating precisely what happens to 
register J and the comparison indicator as a result of the subroutine; state also what 
happens if register I1 is not positive. 


4. [21] Write a subroutine that generalizes MAXN by finding the maximum value of 
Xfa],Xla+r],Xla+2r],..., Xn], where r and n are parameters and a is the smallest 
positive number with a = n (modulo r), namely a = 1 + (n — 1) mod r. Give a special 
entrance for the case r = 1. List the characteristics of your subroutine, as in (4). 

5. [21] Suppose MIX did not have a J-register. Invent a means for subroutine linkage 
that does not use register J, and give an example of your invention by writing a MAX100 
subroutine effectively equivalent to (1). State the characteristics of this subroutine in 
a fashion similar to (4). (Retain MIX’s conventions of self-modifying code.) 

6. [26] Suppose MIX did not have a MOVE operator. Write a subroutine entitled MOVE 
such that the calling sequence ‘JMP MOVE; NOP A,I(F)’ has an effect just the same as 
‘MOVE A,I(F)’ if the latter were admissible. The only differences should be the effect 
on register J and the fact that a subroutine naturally consumes more time and space 
than a hardware instruction does. 


7. [20] Why is self-modifying code now frowned on? 


1.4.2. Coroutines 


Subroutines are special cases of more general program components, called co- 
routines. In contrast to the unsymmetric relationship between a main routine 
and a subroutine, there is complete symmetry between coroutines, which call on 
each other. 
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To understand the coroutine concept, let us consider another way of thinking 
about subroutines. The viewpoint adopted in the previous section was that a 
subroutine merely was an extension of the computer hardware, introduced to 
save lines of coding. This may be true, but another point of view is possible: 
We may consider the main program and the subroutine as a team of programs, 
each member of the team having a certain job to do. The main program, in 
the course of doing its job, will activate the subprogram; the subprogram will 
perform its own function and then activate the main program. We might stretch 
our imagination to believe that, from the subroutine’s point of view, when it 
exits it is calling the main routine; the main routine continues to perform its 
duty, then “exits” to the subroutine. The subroutine acts, then calls the main 
routine again. 

This somewhat far-fetched philosophy actually takes place with coroutines, 
for which it is impossible to distinguish which is a subroutine of the other. 
Suppose we have coroutines A and B; when programming A, we may think of B as 
our subroutine, but when programming B, we may think of A as our subroutine. 
That is, in coroutine A, the instruction ‘JMP B’ is used to activate coroutine B. 
In coroutine B the instruction ‘JMP A’ is used to activate coroutine A again. 
Whenever a coroutine is activated, it resumes execution of its program at the 
point where the action was last suspended. 

The coroutines A and B might, for example, be two programs that play chess. 
We can combine them so that they will play against each other. 

With MIX, such linkage between coroutines A and B is done by including the 
following four instructions in the program: 


A STJ BX B STJ AX 
AX JMP Al BX JMP B1 


(1) 


This requires four machine cycles for transfer of control each way. Initially AX and 
BX are set to jump to the starting places of each coroutine, A1 and B1. Suppose we 
start up coroutine A first, at location A1. When it executes ‘JMP B’ from location 
A2, say, the instruction in location B stores rJ in AX, which then says ‘JMP A2+1’. 
The instruction in BX gets us to location B1, and after coroutine B begins its 
execution, it will eventually get to an instruction ‘JMP A’ in location B2, say. We 
store rJ in BX and jump to location A2+1, continuing the execution of coroutine 
A until it again jumps to B, which stores rJ in AX and jumps to B2+1, etc. 

The essential difference between routine-subroutine and coroutine-coroutine 
linkage, as can be seen by studying the example above, is that a subroutine is 
always initiated at its beginning, which is usually a fixed place; the main routine 
or a coroutine is always initiated at the place following where it last terminated. 

Coroutines arise most naturally in practice when they are connected with 
algorithms for input and output. For example, suppose it is the duty of corou- 
tine A to read cards and to perform some transformation on the input, reducing 
it to a sequence of items. Another coroutine, which we will call B, does further 
processing of these items, and prints the answers; B will periodically call for the 
successive input items found by A. Thus, coroutine B jumps to A whenever it 
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wants the next input item, and coroutine A jumps to B whenever an input item 
has been found. The reader may say, “Well, B is the main program and A is 
merely a subroutine for doing the input.” This, however, becomes less true when 
the process A is very complicated; indeed, we can imagine A as the main routine 
and B as a subroutine for doing the output, and the above description remains 
valid. The usefulness of the coroutine idea emerges midway between these two 
extremes, when both A and B are complicated and each one calls the other in 
numerous places. It is rather difficult to find short, simple examples of coroutines 
that illustrate the importance of the idea; the most useful coroutine applications 
are generally quite lengthy. 

In order to study coroutines in action, let us consider a contrived example. 
Suppose we want to write a program that translates one code into another. The 
input code to be translated is a sequence of alphameric characters terminated 
by a period, such as 

A2B5E3426FGOZYW3210PQ89R. (2) 


This has been punched onto cards; blank columns appearing on these cards are 
to be ignored. The input is to be understood as follows, from left to right: If the 
next character is a digit 0, 1, ..., 9, say n, it indicates (n + 1) repetitions of the 
following character, whether the following character is a digit or not. A nondigit 
simply denotes itself. The output of our program is to consist of the sequence 
indicated in this manner and separated into groups of three characters each, 
until a period appears; the last group may have fewer than three characters. For 
example, (2) should be translated by our program into 


ABB BEE EEE E44 446 66F GZY W22 220 OPQ 999 999 999 R. (3) 


Note that 3426F does not mean 3427 repetitions of the letter F; it means 4 fours 
and 3 sixes followed by F. If the input sequence is ‘1.’, the output is simply ‘.’, 
not ‘..’, because the first period terminates the output. Our program should 
punch the output onto cards, with sixteen groups of three on each card except 
possibly the last. 

To accomplish this translation, we will write two coroutines and a subrou- 
tine. The subroutine, called NEXTCHAR, is designed to find nonblank characters 
of the input, and to put the next such character into register A: 


01 * SUBROUTINE FOR CHARACTER INPUT 


02 READER EQU 16 Unit number of card reader 
03 INPUT ORIG *+16 Place for input cards 

04 NEXTCHAR STJ 9F Entrance to subroutine 

05 JXNZ 3F Initially rX = 0 

06 1H J6N 2F Initially rl6 = 0 

07 IN INPUT(READER) Read next card. 

08 JBUS *(READER) Wait for completion. 

09 ENN6 16 Let rI6 point to the first word. 
10 2H LDX INPUT+16,6 Get the next word of input. 


11 INC6 1 Advance pointer. 
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12 3H ENTA O 

13 SLAX 1 Next character > rA. 
14 9H JANZ * Skip blanks. 

15 JMP NEXTCHAR+1 I 


This subroutine has the following characteristics: 


Calling sequence: JMP NEXTCHAR. 

Entry conditions: rX = characters yet to be used; rl6 points to next word, or 
rl6 = 0 indicating that a new card must be read. 

Exit conditions: rA = next nonblank character of input; rX and rI6 are set for 
next entry to NEXTCHAR. 


Our first coroutine, called IN, finds the characters of the input code with 
the proper replication. It begins initially at location IN1: 


16 * FIRST COROUTINE 


17 2H INCA 30 Nondigit found 

18 JMP OUT Send it to OUT coroutine. 
19 IN1 JMP NEXTCHAR Get character. 

20 DECA 30 

21 JAN 2B Is it a letter? 

22 CMPA =10= 

28 JGE 2B Is it a special character? 
24 STA *+1(0:2) Digit n found 

25 ENTS * rI5 + n. 

26 JMP NEXTCHAR Get next character. 

27 JMP OUT Send it to OUT coroutine. 
28 DEC5 1 Decrease n by 1. 

29 JSNN *-2 Repeat if necessary. 

30 JMP IN1 Begin new cycle. J 


(Recall that in MIX’s character code, the digits 0-9 have codes 30-39.) This 
coroutine has the following characteristics: 


Calling sequence: JMP IN. 
Exit conditions (when 
jumping to OUT): rA = next character of input with proper replication; rI4 


unchanged from its value at entry. 
Entry conditions 


(upon return): rA, rX, rl5, rl6 should be unchanged from their values 
at the last exit. 
The other coroutine, called OUT, puts the code into three-character groups 
and punches the cards. It begins initially at OUT1: 


31 * SECOND COROQUTINE 


32 ALF Constant used for blanking 
83 QUTPUT ORIG *+16 Buffer area for answers 
34 PUNCH EQU 17 Unit number for card punch 


85 QUT1 ENT4 -16 Start new output card. 


1.4.2 


36 ENT1 
37 MOVE 
38 1H JMP 
39 STA 
40 CMPA 
41 JE 
42 JMP 
43 STA 
44 CMPA 
45 JE 
46 JMP 
41 STA 
48 CMPA 
49 JE 
50 INC4 
51 J4N 
52 9H OUT 
53 JBUS 
54 JNE 
55 HLT 


56 PERIOD ALF 


OUTPUT 

-1,1(16) 

IN 
OUTPUT+16,4(1:1) 
PERIOD 

oF 

IN 
OUTPUT+16,4(2:2) 
PERIOD 

oF 

IN 

OUTPUT+16 ,4(3:3) 
PERIOD 

OF 

1 

1B 

OUTPUT (PUNCH) 

* (PUNCH) 

OUT 1 


UUW + 
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Set output area to blanks. 
Get next translated character. 
Store it in the (1:1) field. 

Is it ‘.’? 


If not, get another character. 
Store it in the (2:2) field. 
Is it ‘.’? 


If not, get another character. 
Store it in the (3:3) field. 
Is it ‘.’? 


Move to next word in output buffer. 
End of card? 
If so, punch it. 
Wait for completion. 
Return for more, unless 
‘.’ was sensed. 


This coroutine has the following characteristics: 


Calling sequence: 


JMP OUT. 


Exit conditions (when 


jumping to IN): 


Entry conditions 


(upon return): 


rA, rX, rI5, rl6 unchanged from their value at entry; rll 


possibly affected; previous character recorded in output. 


rA = next character of input with proper replication; rI4 
unchanged from its value at the last exit. 


To complete the program, we need to write the coroutine linkage (see (1)) 
and to provide the proper initialization. Initialization of coroutines tends to be 
a little tricky, although not really difficult. 


57 

58 START ENT6 
59 ENTX 
60 JMP 
61 OUT STJ 
62 OUTX JMP 
63 IN STJ 
64 INX JMP 
65 END 


* INITIALIZATION AND LINKAGE 


0 Initialize rl6 for NEXTCHAR. 

0 Initialize rX for NEXTCHAR. 
OUT1 Start with OUT (see exercise 2). 
INX Coroutine linkage 

OUT1 

OUTX 

IN1 

START I 


This completes the program. The reader should study it carefully, noting in 
particular how each coroutine can be written independently as though the other 
coroutine were its subroutine. 
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The entry and exit conditions for the IN and OUT coroutines mesh perfectly in 
the program above. In general, we would not be so fortunate, and the coroutine 
linkage would also include instructions for loading and storing appropriate regis- 
ters. For example, if OUT would destroy the contents of register A, the coroutine 
linkage would become 


OUT STJ INX 


STA HOLDA Store A when leaving IN. 
OUTX JMP OUT1 


IN STJ OUTX (4) 
LDA HOLDA Restore A when leaving OUT. 
INX JMP IN1 I 


There is an important relation between coroutines and multipass algorithms. 
For example, the translation process we have just described could have been done 
in two distinct passes: We could first have done just the IN coroutine, applying 
it to the entire input and writing each character with the proper amount of 
replication onto magnetic tape. After this was finished, we could rewind the 
tape and then do just the OUT coroutine, taking the characters from tape in 
groups of three. This would be called a “two-pass” process. (Intuitively, a 
“pass” denotes a complete scan of the input. This definition is not precise, 
and in many algorithms the number of passes taken is not at all clear; but the 
intuitive concept of “pass” is useful in spite of its vagueness.) 

Figure 22(a) illustrates a four-pass process. Quite often we will find that 
the same process can be done in just one pass, as shown in part (b) of the figure, 
if we substitute four coroutines A, B, C, D for the respective passes A, B, C, D. 
Coroutine A will jump to B when pass A would have written an item of output 
on tape 1; coroutine B will jump to A when pass B would have read an item of 
input from tape 1, and B will jump to C when pass B would have written an item 
of output on tape 2; etc. UNIX® users will recognize this as a “pipe,” denoted by 
‘PassA | PassB | PassC | PassD’. The programs for passes B, C, and D are 
sometimes referred to as “filters.” 

Conversely, a process done by n coroutines can often be transformed into an 
n-pass process. Due to this correspondence it is worthwhile to compare multipass 
algorithms with one-pass algorithms. 


a) Psychological difference. A multipass algorithm is generally easier to create 
and to understand than a one-pass algorithm for the same problem. Breaking a 
process down into a sequence of small steps that happen one after the other is 
easier to comprehend than an involved process in which many transformations 
take place simultaneously. 

Also, if a very large problem is being tackled and if many people are to 
co-operate in producing a computer program, a multipass algorithm provides a 
natural way to divide up the job. 

These advantages of a multipass algorithm are present in coroutines as well, 
since each coroutine can be written essentially separate from the others, and the 
linkage makes an apparently multipass algorithm into a single-pass process. 


Re 
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Pass A Coroutine A 


Pass B Coroutine B 


Pass C Coroutine C 


Coroutine D (upu) 


Fig. 22. Passes: (a) a four-pass algorithm, and (b) a one-pass algorithm. 


Pass D 


HH 


b) Time difference. The time required to pack, write, read, and unpack the 
intermediate data that flows between passes (for example, the information on 
tapes in Fig. 22) is avoided in a one-pass algorithm. For this reason, a one-pass 
algorithm will be faster. 


c) Space difference. The one-pass algorithm requires space to hold all the 
programs in memory simultaneously, while a multipass algorithm requires space 
for only one at a time. This requirement may affect the speed, even to a greater 
extent than indicated in statement (b). For example, many computers have a 
limited amount of “fast memory” and a larger amount of slower memory; if each 
pass just barely fits into the fast memory, the result will be considerably faster 
than if we use coroutines in a single pass (since the use of coroutines would 
presumably force most of the program to appear in the slower memory or to be 
repeatedly swapped in and out of fast memory). 

Occasionally there is a need to design algorithms for several computer con- 
figurations at once, some of which have larger memory capacity than others. In 
such cases it is possible to write the program in terms of coroutines, and to let 
the memory size govern the number of passes: Load together as many coroutines 
as feasible, and supply input or output subroutines for the missing links. 


Although this relationship between coroutines and passes is important, we 
should keep in mind that coroutine applications cannot always be split into 
multipass algorithms. If coroutine B gets input from A and also sends back 
crucial information to A, as in the example of chess play mentioned earlier, the 
sequence of actions can’t be converted into pass A followed by pass B. 

Conversely, it is clear that some multipass algorithms cannot be converted 
to coroutines. Some algorithms are inherently multipass; for example, the second 
pass may require cumulative information from the first pass (like the total 


v 
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number of occurrences of a certain word in the input). There is an old joke 
worth noting in this regard: 


Little old lady, riding a bus. “Little boy, can you tell me how to get off 
at Pasadena Street?” 
Little boy. “Just watch me, and get off two stops before I do.” 


(The joke is that the little boy gives a two-pass algorithm.) 

So much for multipass algorithms. We will see further examples of coroutines 
in numerous places throughout this book, for example, as part of the buffering 
schemes in Section 1.4.4. Coroutines also play an important role in discrete 
system simulation; see Section 2.2.5. The important idea of replicated coroutines 
is discussed in Chapter 8, and some interesting applications of this idea may be 
found in Chapter 10. 


EXERCISES 

1. [10] Explain why short, simple examples of coroutines are hard for the author of 
a textbook to find. 

2. [20] The program in the text starts up the OUT coroutine first. What would 
happen if IN were the first to be executed—that is, if line 60 were changed from 
‘JMP OUT1’ to ‘JMP IN1’? 

3. [20] True or false: The three ‘CMPA PERIOD’ instructions within OUT may all be 
omitted, and the program would still work. (Look carefully.) 

4. [20] Show how coroutine linkage analogous to (1) can be given for real-life com- 
puters you are familiar with. 


5. [15] Suppose both coroutines IN and OUT want the contents of register A to remain 
untouched between exit and entry; in other words, assume that wherever the instruction 
‘JMP IN’ occurs within OUT, the contents of register A are to be unchanged when control 
returns to the next line, and make a similar assumption about ‘JMP OUT’ within IN. 
What coroutine linkage is needed? (Compare with (4).) 

6. [22] Give coroutine linkage analogous to (1) for the case of three coroutines, A, 
B, and C, each of which can jump to either of the other two. (Whenever a coroutine is 
activated, it begins where it last left off.) 

7. [30] Write a MIX program that reverses the translation done by the program in the 
text; that is, your program should convert cards punched like (3) into cards punched 
like (2). The output should be as short a string of characters as possible, so that the 
zero before the Z in (2) would not really be produced from (3). 


1.4.3. Interpretive Routines 


In this section we will investigate a common type of computer program, the 
interpretive routine (which will be called interpreter for short). An interpretive 
routine is a computer program that performs the instructions of another pro- 
gram, where the other program is written in some machine-like language. By a 
machine-like language, we mean a way of representing instructions, where the 
instructions typically have operation codes, addresses, etc. (This definition, like 
most definitions of today’s computer terms, is not precise, nor should it be; we 
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cannot draw the line exactly and say just which programs are interpreters and 
which are not.) 

Historically, the first interpreters were built around machine-like languages 
designed specially for simple programming; such languages were easier to use 
than a real machine language. The rise of symbolic languages for programming 
soon eliminated the need for interpretive routines of that kind, but interpreters 
have by no means begun to die out. On the contrary, their use has continued 
to grow, to the extent that an effective use of interpretive routines may be 
regarded as one of the essential characteristics of modern programming. The 
new applications of interpreters are made chiefly for the following reasons: 


a) a machine-like language is able to represent a complicated sequence of deci- 
sions and actions in a compact, efficient manner; and 


b) such a representation provides an excellent way to communicate between 
passes of a multipass process. 

In such cases, special purpose machine-like languages are developed for use in 
a particular program, and programs in those languages are often generated only 
by computers. (Today’s expert programmers are also good machine designers, as 
they not only create an interpretive routine, they also define a virtual machine 
whose language is to be interpreted.) 

The interpretive technique has the further advantage of being relatively 
machine-independent — only the interpreter must be rewritten when changing 
computers. Furthermore, helpful debugging aids can readily be built into an 
interpretive system. 

Examples of interpreters of type (a) appear in several places later in this 
series of books; see, for example, the recursive interpreter in Chapter 8 and the 
“Parsing Machine” in Chapter 10. We typically need to deal with a situation in 
which a great many special cases arise, all similar, but having no really simple 
pattern. 

For example, consider writing an algebraic compiler in which we want to gen- 
erate efficient machine-language instructions that add two quantities together. 
There might be ten classes of quantities (constants, simple variables, temporary 
storage locations, subscripted variables, the contents of an accumulator or index 
register, fixed or floating point, etc.) and the combination of all pairs yields 100 
different cases. A long program would be required to do the proper thing in 
each case. The interpretive solution to this problem is to make up an ad hoc 
language whose “instructions” fit in one byte. Then we simply prepare a table 
of 100 “programs” in this language, where each program ideally fits in a single 
word. The idea is then to pick out the appropriate table entry and to perform 
the program found there. This technique is simple and efficient. 

An example interpreter of type (b) appears in the article “Computer-Drawn 
Flowcharts” by D. E. Knuth, CACM 6 (1963), 555-563. In a multipass program, 
the earlier passes must transmit information to the later passes. This information 
is often transmitted most efficiently in a machine-like language, as a set of 
instructions for the later pass; the later pass is then nothing but a special purpose 
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interpretive routine, and the earlier pass is a special purpose “compiler.” This 
philosophy of multipass operation may be characterized as telling the later pass 
what to do, whenever possible, rather than simply presenting it with a lot of 
facts and asking it to figure out what to do. 

Another example of a type-(b) interpreter occurs in connection with com- 
pilers for special languages. If the language includes many features that are not 
easily done on the machine except by subroutine, the resulting object programs 
will be very long sequences of subroutine calls. This would happen, for example, 
if the language were concerned primarily with multiple-precision arithmetic. In 
such a case the object program would be considerably shorter if it were expressed 
in an interpretive language. See, for example, the book ALGOL 60 Implementa- 
tion, by B. Randell and L. J. Russell (New York: Academic Press, 1964), which 
describes a compiler to translate from ALGOL 60 into an interpretive language, 
and which also describes the interpreter for that language; and see “An ALGOL 
60 Compiler,” by Arthur Evans, Jr., Ann. Rev. Auto. Programming 4 (1964), 
87-124, for examples of interpretive routines used within a compiler. The rise of 
microprogrammed machines and of special-purpose integrated circuit chips has 
made this interpretive approach even more valuable. 

The TẸX program, which produced the pages of the book you are now 
reading, converted a file that contained the text of this section into an interpretive 
language called DVI format, designed by D. R. Fuchs in 1979. [See D. E. Knuth, 
TEX: The Program (Reading, Mass.: Addison-Wesley, 1986), Part 31.] The DVI 
file that TEX produced was then processed by an interpreter called dvips, written 
by T. G. Rokicki, and converted to a file of instructions in another interpretive 
language called PostScript® [Adobe Systems Inc., PostScript Language Reference 
Manual, 2nd edition (Reading, Mass.: Addison-Wesley, 1990)]. The PostScript 
file was sent to the publisher, who sent it to a commercial printer, who used 
a PostScript interpreter to produce printing plates. This three-pass operation 
illustrates interpreters of type (b); TẸX itself also includes a small interpreter of 
type (a) to process the so-called ligature and kerning information for characters 
of each font of type [Tsx: The Program, §545]. 

There is another way to look at a program written in interpretive language: 
It may be regarded as a series of subroutine calls, one after another. Such a pro- 
gram may in fact be expanded into a long sequence of calls on subroutines, and, 
conversely, such a sequence can usually be packed into a coded form that is read- 
ily interpreted. The advantages of interpretive techniques are the compactness of 
representation, the machine independence, and the increased diagnostic capabil- 
ity. An interpreter can often be written so that the amount of time spent in inter- 
pretation of the code itself and branching to the appropriate routine is negligible. 


1.4.3.1. A MIX simulator. When the language presented to an interpretive 
routine is the machine language of another computer, the interpreter is often 
called a simulator (or sometimes an emulator). 

In the author’s opinion, entirely too much programmers’ time has been 
spent in writing such simulators and entirely too much computer time has been 
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wasted in using them. The motivation for simulators is simple: A computer 
installation buys a new machine and still wants to run programs written for 
the old machine (rather than rewriting the programs). However, this usually 
costs more and gives poorer results than if a special task force of programmers 
were given temporary employment to do the reprogramming. For example, the 
author once participated in such a reprogramming project, and a serious error 
was discovered in the original program, which had been in use for several years; 
the new program worked at five times the speed of the old, besides giving the 
right answers for a change! (Not all simulators are bad; for example, it is usually 
advantageous for a computer manufacturer to simulate a new machine before it 
has been built, so that software for the new machine may be developed as soon as 
possible. But that is a very specialized application.) An extreme example of the 
inefficient use of computer simulators is the true story of machine A simulating 
machine B running a program that simulates machine C! This is the way to 
make a large, expensive computer give poorer results than its cheaper cousin. 

In view of all this, why should such a simulator rear its ugly head in this 
book? There are two reasons: 


a) The simulator we will describe below is a good example of a typical interpre- 
tive routine; the basic techniques employed in interpreters are illustrated here. 
It also illustrates the use of subroutines in a moderately long program. 

b) We will describe a simulator of the MIX computer, written in (of all things) 
the MIX language. This will facilitate the writing of MIX simulators for most 
computers, which are similar; the coding of our program intentionally avoids 
making heavy use of MIX-oriented features. A MIX simulator will be of advantage 
as a teaching aid in conjunction with this book and possibly others. 


Computer simulators as described in this section should be distinguished 
from discrete system simulators. Discrete system simulators are important pro- 
grams that will be discussed in Section 2.2.5. 

Now let’s turn to the task of writing a MIX simulator. The input to our 
program will be a sequence of MIX instructions and data, stored in locations 
0000-3499. We want to mimic the precise behavior of MIX’s hardware, pretending 
that MIX itself is interpreting those instructions; thus, we want to implement 
the specifications that were laid down in Section 1.3.1. Our program will, for 
example, maintain a variable called AREG that will hold the magnitude of the 
simulated A-register; another variable, SIGNA, will hold the corresponding sign. 
A variable called CLOCK will record how many MIX units of simulated time have 
elapsed during the simulated program execution. 

The numbering of MIX’s instructions LDA, LD1, ..., LDX and other similar 
commands suggests that we keep the simulated contents of these registers in 
consecutive locations, as follows: 


AREG, I1REG, I2REG, I3REG, I4REG, ISREG, I6REG, XREG, JREG, ZERO. 


Here ZERO is a “register” filled with zeros at all times. The positions of JREG and 
ZERO are suggested by the op-code numbers of the instructions STJ and STZ. 
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In keeping with our philosophy of writing the simulator as though it were not 
really done with MIX hardware, we will treat the signs as independent parts of 
a register. For example, many computers cannot represent the number “minus 
zero”, while MIX definitely can; therefore we will always treat signs specially 
in this program. The locations AREG, I1REG, ..., ZERO will always contain the 
absolute values of the corresponding register contents; another set of locations in 
our program, called SIGNA, SIGN1, ..., SIGNZ will contain +1 or —1, depending 
on whether the sign of the corresponding register is plus or minus. 

An interpretive routine generally has a central control section that is called 
into action between interpreted instructions. In our case, the program transfers 
to location CYCLE at the end of each simulated instruction. 

The control routine does the things common to all instructions, unpacks the 
instruction into its various parts, and puts the parts into convenient places for 
later use. The program below sets 


rIl6 = location of the next instruction; 
rI5 = M (address of the present instruction, plus indexing); 
rl4 = operation code of the present instruction; 
rI3 = F-field of the present instruction; 
INST = the present instruction. 


Program M. 

001 * MIX SIMULATOR 

002 ORIG 3500 Simulated memory is in locations 0000 up. 
003 BEGIN STZ TIME(0:2) 

004 STZ OVTOG OVTOG is the simulated overflow toggle. 
005 STZ COMPI COMPI, +1 or 0, is comparison indicator. 
006 ENT6 O Take first instruction from location zero. 
007 CYCLE LDA CLOCK Beginning of control routine: 

008 TIME INCA 0 This address is set to the execution time 
009 STA CLOCK of the previous instruction (see line 033). 
010 LDA 0,6 rA & instruction to simulate. 

011 STA INST 

012 INC6 1 Advance the location counter. 

013 LDX INST(1:2) Get absolute value of the address. 

014 SLAX 5 Attach sign to the address. 

015 STA M 

016 LD2 INST(3:3) Examine the index field. 

017 J2Z 1F Is it zero? 

018 DEC2 6 

019 J2P INDEXERROR Illegal index specified? 

020 LDA SIGN6,2 Get sign of the index register. 

021 LDX I6REG,2 Get magnitude of the index register. 

022 SLAX 5 Attach the sign. 

023 ADD M Do signed addition for indexing. 

024 CMPA ZERO(1:3) Is the result too large? 

025 JNE ADDRERROR If so, simulate an error. 

026 STA M Otherwise the address has been found. 
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027 1H 
028 
029 
030 
031 
032 
033 
034 
035 
036 


LD3 
LD5 
LD4 
DEC4 
J4P 
LDA 
STA 
LD2 
JNOV 
JMP 


INST (4:4) 
M 

INST (5:5) 
63 
OPERROR 


OPTABLE,4(4:4) 


TIME (0:2) 


OPTABLE, 4(0:2) 


0,2 
0,2 
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rl3 + F-field. 
rI5 + M. 
rl4 + C-field. 


Is the op code > 64? 
Get execution time from the table. 


Get address of the proper routine. 
Jump to operator. 
(Protect against overflows.) | 


The reader’s attention is called particularly to lines 034-036: A “switching 
table” of the 64 operators is part of the simulator, allowing it to jump rapidly to 
the correct routine for the current instruction. This is an important time-saving 
technique (see exercise 1.3.2—9). 

The 64-word switching table, called OPTABLE, gives also the execution time 
for the various operators; the following lines indicate the contents of that table: 


037 
038 
039 
040 
041 
042 
043 
044 
045 
046 


051 
052 
053 
054 


060 
061 


069 
070 
O71 
O72 
O78 
O14 
O75 
O76 
O77 


084 
085 


NOP 
ADD 
SUB 
MUL 
DIV 
HLT 
SLA 
MOVE 
LDA 
LD1 
LD6 
LDX 
LDAN 
LD1N 
LDXN 
STA 
STJ 
STZ 
JBUS 
I0C 
IN 
OUT 
JRED 
JMP 
JAP 
JXP 
INCA 


CYCLE (1) 
ADD (2) 
SUB (2) 
MUL(10) 
DIV(12) 
SPEC (10) 
SHIFT (2) 
MOVE (1) 
LOAD (2) 
LOAD, 1 (2) 


LOAD, 1 (2) 
LOAD (2) 
LOADN (2) 
LOADN, 1(2) 


LOADN (2) 
STORE (2) 


STORE (2) 
STORE (2) 
JBUS (1) 
TOC(1) 
IN(1) 

OUT (1) 
JRED (1) 
JUMP (1) 
REGJUMP (1) 


REG JUMP (1) 
ADDROP (1) 


Operation code table; 
typical entry is 
“OP routine(time)” 
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086 INC1 ADDROP,1(1) 
092 INCX ADDROP (1) 
093 CMPA COMPARE(2) 


100 OPTABLE CMPX COMPARE(2) J 


(The entries for operators LDi, LDiN, and INCi have an additional ‘,1’ to set the 
(3:3) field nonzero; this is used below in lines 289-290 to indicate the fact that 
the size of the quantity within the corresponding index register must be checked 
after simulating these operations.) 

The next part of our simulator program merely lists the locations used to 
contain the contents of the simulated registers: 


101 AREG CON O Magnitude of A-register 

102 TI1REG CON O Magnitude of index registers 
107 I6REG CON O 

108 XREG CON O Magnitude of X-register 

109 JREG CON O Magnitude of J-register 

110 ZERO CON O Constant zero, for ‘STZ’ 

111 SIGNA CON 1. Sign of A-register 

112 SIGN1 CON 1 Sign of index registers 

117 SIGN6 CON 1 

118 SIGNX CON 1 Sign of X-register 

119 SIGNI CON 1 Sign of J-register 

120 SIGNZ CON 1 Sign stored by ‘STZ’ 

121 INST CON O Instruction being simulated 
122 COMPI CON O Comparison indicator 

123 OVTOG CON O Overflow toggle 

124 CLOCK CON O Simulated execution time J] 


Now we will consider three subroutines used by the simulator. First comes 
the MEMORY subroutine: 


Calling sequence: JMP MEMORY. 

Entry conditions: rI5 = valid memory address (otherwise the subroutine will 
jump to MEMERROR). 

Exit conditions: rX = sign of word in memory location rI5; rA = magnitude of 
word in memory location rI5. 


125 * SUBROUTINES 


126 MEMORY STJ 9F Memory fetch subroutine: 

127 J5N MEMERROR 

128 CMP5 =BEGIN= The simulated memory is in 

129 JGE MEMERROR locations 0000 to BEGIN — 1. 
130 LDX 0,5 

131 ENTA 1 


132 SRAX 5 rX & sign of word. 
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133 LDA 0,5(1:5) rA + magnitude of word. 
134 9H JMP x Exit. | 


The FCHECK subroutine processes a partial field specification, making sure 
that it has the form 8L +R with L<R<5. 


Calling sequence: JMP FCHECK. 

Entry conditions: rI3 = valid field specification (otherwise the subroutine will 
jump to FERROR). 

Exit conditions: rA =rll=L,rX=R. 


185 FCHECK STJ 9F Field check subroutine: 
186 ENTA O 

137 ENTX 0,3 rAX + field specification. 
138 DIV =8= rA eL, rXeR. 

139 CMPX =5= Is R > 5? 

140 JG FERROR 

141 STX R 

142 STA L 

143 LD1 L ril + L. 

144 CMPA R 

145 9H JLE * Exit unless L > R. 


146 JMP FERROR I 


The last subroutine, GETV, finds the quantity V (namely, the appropriate 
field of location M) used in various MIX operators, as defined in Section 1.3.1. 


Calling sequence: JMP GETV. 

Entry conditions: r15 = valid memory address; rI3 = valid field. (If invalid, an 
error will be detected as above.) 

Exit conditions: rA = magnitude of V; rX = sign of V; rll = L; rl2 = —R. 

Second entrance: JMP GETAV, used only in comparison operators to extract a 
field from a register. 


147 GETAV STJ 9F Special entrance, see line 300. 
148 JMP 1F 

149 GETV STJ 9F Subroutine to find V: 

150 JMP FCHECK Process the field and set rll + L. 
151 JMP MEMORY rA <+ memory magnitude, rX < sign. 
152 1H J1Z 2F Is the sign included in the field? 
153 ENTX 1 If not, set the sign positive. 

154 SLA =1,1 Zero out all bytes to the left 

155 SRA -1,1 of the field. 

156 2H LD2N R Shift right into the 

157 SRA 5,2 proper position. 

158 9H JMP * Exit. I 


Now we come to the routines for each individual operator. These routines 
are given here for completeness, but the reader should study only a few of them 
unless there’s a compelling reason to look closer; the SUB and JUMP operators are 
recommended as typical examples for study. Notice the way in which routines 
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for similar operations can be neatly combined, and notice how the JUMP routine 
uses another switching table to govern the type of jump. 


159 * INDIVIDUAL OPERATORS 


160 ADD JMP GETV Get the value of V in rA and rX. 
161 ENT1 0 rll + index of simulated rA. 

162 JMP INC Go to the “increase” routine. 
163 SUB JMP GETV Get the value of V in rA and rX. 
164 ENT1 O rll + index of simulated rA. 

165 JMP DEC Go to the “decrease” routine. 
166 * 

167 MUL JMP GETV Get the value of V in rA and rX. 
168 CMPX SIGNA Are signs the same? 

169 ENTX 1 

170 JE *+2 Set rX to the result sign. 

171 ENNX 1 

172 STX SIGNA Put it in both simulated registers. 
178 STX SIGNX 

174 MUL AREG Multiply the operands. 

175 JMP STOREAX Store the magnitudes. 

176 * 

177 DIV LDA SIGNA Set the sign of the remainder. 
178 STA SIGNX 

179 JMP GETV Get the value of V in rA and rX. 
180 CMPX SIGNA Are signs the same? 

181 ENTX 1 

182 JE *+2 Set rX to the result sign. 

183 ENNX 1 

184 STX SIGNA Put it in the simulated rA. 

185 STA TEMP 

186 LDA AREG Divide the operands. 

187 LDX XREG 

188 DIV TEMP 

189 STOREAX STA AREG Store the magnitudes. 

190 STX XREG 

191 OVCHECK JNOV CYCLE Did overflow just occur? 

192 ENTX 1 If so, set the simulated 

193 STX OVTOG overflow toggle on. 

194 JMP CYCLE Return to control routine. 

195 * 

196 LOADN JMP GETV Get the value of V in rA and rX. 
197 ENT1 47,4 rll + C — 16; indicates register. 
198 LOADN1 STX TEMP Negate the sign. 

199 LDXN TEMP 

200 JMP LOAD1 Change LOADN to LOAD. 

201 LOAD JMP GETV Get the value of V in rA and rX. 
202 ENT1 55,4 rll + C — 8, indicates register. 
208 LOAD1 STA AREG,1 Store the magnitude. 


204 STX SIGNA,1 Store the sign. 


1H 


JMP 


JTABLE 
JOV 


JNOV 


LE 
LS 


NE 
GR 


GE 
EQ 


Ea 


YQORPuunyn7ungnw~ununnunny 


How 
Hur ao) 


E E 
‘Uv y 


E 
‘U 


SIZECHK 


FCHECK 
MEMORY 

1F 

1 
SIGNA+39,4 
R 

5,2 
AREG+39,4 


FERROR 
COMPI 
JTABLE,3 
JREG 
JSJ 
JOV 
JNOV 
LS 

EQ 

GR 

GE 

NE 
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Check if the magnitude is too large. 


ril + L. 
Get contents of memory location. 
Is the sign included in the field? 
If so, change L to 1 

and “store” the register’s sign. 
rI2 + —R. 
Save the area to the field’s right. 
Insert register in the field. 


rī2 + —L. 
Restore the area to the field’s left. 


Attach the sign. 
Store in memory. 
Return to control routine. 


Jump operators: 

Is F too large? 

rA + comparison indicator. 
Jump to appropriate routine. 
Set the simulated J-register. 


End of the jump table 
Check whether to jump on 
overflow. 


Get complement of overflow toggle. 
Shut off overflow toggle. 

Jump. 

Don’t jump. 

Jump if rA zero or negative. 
Jump if rA negative. 

Don’t jump. 

Jump if rA negative or positive. 
Jump if rA positive. 

Don’t jump. 

Jump if rA positive or zero. 
Jump if rA zero. 

Don’t jump. 
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254 
255 
256 
257 
258 
259 
260 
261 
262 
268 
264 
265 
266 
267 
268 
269 
270 
271 
272 
278 
274 
275 
276 
277 
278 
279 
280 
281 
282 
288 
284 
285 
286 
287 
288 
289 
290 
291 
292 
298 
294 
295 
296 
297 
298 
299 
300 
301 
302 
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JSJ 


* 
REGJUMP 


* 
ADDROP 


1H 
DEC 


INC 


1H 
2H 
SIZECHK 


* 
COMPARE 


JMP 
ENT6 
JMP 


LDA 
JAZ 
LDA 
DEC3 
J3NP 
JMP 


fa 
aw 
HU 
Ps 


js) 


MEMORY 
0,5 
CYCLE 


AREG+23,4 
*+2 
SIGNA+23 ,4 
5 

JTABLE,3 
FERROR 


M(1:5) 
15,4 
1F,3 
INC 
DEC 
LOAD1 
LOADN1 
TEMP 
TEMP 
SIGNA, 1 
1F 
AREG, 1 
QF 
SIGNA, 1 
QF 
AREG, 1 
AREG,1(1:5) 


OPTABLE, 4(3:3) 


OVCHECK 
ZERO (1:3) 
CYCLE 
SIZEERROR 


GETV 

5 

V 
XREG,4 
SIGNX,4 
GETAV 

5 

V 
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Check for valid memory address. 
Simulate a jump. 
Return to main control routine. 


Register jumps: 
Is register zero? 
If not, put sign into rA. 


Change to a conditional JMP, unless 
the F-specification is too large. 


Address transfer operators: 
Is F too large? 


Find the sign of M. 


rX + sign of M. 

rA + magnitude of M. 
rI1 indicates the register. 
Four-way jump. 

Increase. 

Decrease. 

Enter. 

Enter negative. 

Reverse the sign. 

Reduce DEC to INC. 
Addition routine: 

Are signs the same? 

No; subtract magnitudes. 
Sign change needed? 
Change the register’s sign. 


Add magnitudes. 
Store magnitude of the result. 
Have we just loaded an 
index register? 
If so, make sure that the result 
fits in two bytes. 


Get the value of V in rA and rX. 
Attach the sign. 


Get field F of the appropriate register. 


Attach the sign. 
Compare (note that —0 = +0). 
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303 STZ COMPI Set comparison indicator to 
304 JE CYCLE either zero, plus one, 
305 ENTA 1 or minus one. 

306 JG *+2 

307 ENNA 1 

308 STA COMPI 

309 JMP CYCLE Return to control routine. 
310 * 

311 END BEGIN J 


The code above adheres to a subtle rule that was stated in Section 1.3.1: The 
instruction ‘ENTA -0 loads minus zero into register A, as does ‘ENTA -5,1’ when 
index register 1 contains +5. In general, when M is zero, ENTA loads the sign of the 
instruction and ENNA loads the opposite sign. The need to specify this condition 
was overlooked when the author prepared his first draft of Section 1.3.1; such 
questions usually come to light only when a computer program is being written 
to follow the rules. 

In spite of its length, the program above is incomplete in several respects: 


) It does not recognize floating point operations. 

) The coding for operation codes 5, 6, and 7 has been left as an exercise. 

) The coding for input-output operators has been left as an exercise. 

) No provision has been made for loading simulated programs (see exercise 4). 
) The error routines 


INDEXERROR, ADDRERROR, OPERROR, MEMERROR, FERROR, SIZEERROR 


have not been included; they handle error conditions that are detected in 
the simulated program. 

f) There is no provision for diagnostic facilities. (A useful simulator should, 
for example, make it possible to print out the register contents as a program 
is being executed.) 


EXERCISES 


1. [14] Study all the uses of the FCHECK subroutine in the simulator program. Can 
you suggest a better way to organize the code? (See step 3 in the discussion at the end 
of Section 1.4.1.) 

2. [20] Write the SHIFT routine, which is missing from the program in the text 
(operation code 6). 

3. [22] Write the MOVE routine, which is missing from the program in the text 
(operation code 7). 

4. [14] Change the program in the text so that it begins as though MIX’s “GO button” 
had been pushed (see exercise 1.3.1—26). 

5. [24] Determine the time required to simulate the LDA and ENTA operators, com- 
pared with the actual time for MIX to execute these operators directly. 

6. [28] Write programs for the input-output operators JBUS, IOC, IN, OUT, and JRED, 
which are missing from the program in the text, allowing only units 16 and 18. Assume 
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that the operations “read-card” and “skip-to-new-page” take T = 10000u, while “print- 
line” takes T = 7500u. [Note: Experience shows that the JBUS instruction should be 
simulated by treating ‘JBUS *’ as a special case; otherwise the simulator seems to stop!] 

> 7. [32] Modify the solutions of the previous exercise in such a way that execution 
of IN or OUT does not cause I/O transmission immediately; the transmission should 
take place after approximately half of the time required by the simulated devices has 
elapsed. (This will prevent a frequent student error, in which IN and OUT are used 
improperly.) 

8. [20] True or false: Whenever line 010 of the simulator program is executed, we 
have 0 < rl6 < BEGIN. 


*1.4.3.2. Trace routines. When a machine is being simulated on itself (as 
MIX was simulated on MIX in the previous section) we have the special case of 
a simulator called a trace or monitor routine. Such programs are occasionally 
used to help in debugging, since they print out a step-by-step account of how 
the simulated program behaves. 

The program in the preceding section was written as though another com- 
puter were simulating MIX. A quite different approach is used for trace programs; 
we generally let registers represent themselves and let the operators perform 
themselves. In fact, we usually contrive to let the machine execute most of 
the instructions by itself. The chief exception is a jump or conditional jump 
instruction, which must not be executed without modification, since the trace 
program must remain in control. Each machine also has idiosyncratic features 
that make tracing more of a challenge; in MIX’s case, the J-register presents the 
most interesting problem. 

The trace routine given below is initiated when the main program jumps to 
location ENTER, with register J set to the address for starting to trace and reg- 
ister X set to the address where tracing should stop. The program is interesting 
and merits careful study. 


01 * TRACE ROUTINE 


02 ENTER STX TEST(0:2) Set the exit location. 

03 STX LEAVEX(0:2) 

04 STA AREG Save the contents of rA. 

05 STJ JREG Save the contents of rJ. 

06 LDA JREG(0:2) Get the start location for tracing. 
07 CYCLE STA PREG(0:2) Store the location of the next instruction. 
08 TEST DECA * Is it the exit location? 

09 JAZ LEAVE 

10 PREG LDA * Get the next instruction. 

1 STA INST Copy it. 

12 SRA 2 

13 STA INST1(0:3) Store the address and index parts. 
14 LDA INST(5:5) Get the operation code, C. 

15 DECA 38 

16 JANN 1F Is C > 38 (JRED)? 

17 INCA 6 
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18 JANZ 2F Is C Æ 32 (STJ)? 

19 LDA INST(0:4) 

20 STA *+2(0:4) Change STJ to STA. 

21 JREG ENTA * rA + simulated rJ contents. 

22 STA * 

23 JMP INCP 

24 2H DECA 2 

25 JANZ 2F Is C Æ 34 (JBUS)? 

26 JMP 3F 

27 1H DECA 9 Test for jump instructions. 

28 JAP 2F Is C > 47 (JXNP)? 

29 3H LDA 8F(0:3) We detected a jump instruction; 

30 STA INST(0:3) change its address to ‘JUMP’. 

81 2H LDA AREG Restore register A. 

32 * All registers except J now have proper 
33 * values with respect to the external program. 
34 INST NOP * The instruction is executed. 

35 STA AREG Store register A again. 


36 INCP LDA PREG(0:2) Move to the next instruction. 
37 INCA 1 


38 JMP CYCLE 

39 8H JSJ JUMP Constant for lines 29 and 40 
40 JUMP LDA 8B(4:5) A jump has occurred. 

41 SUB INST(4:5) Was it JSJ? 

42 JAZ *+4 

43 LDA PREG(0:2) If not, update the simulated 
44 INCA 1 J-register. 

45 STA JREG(0:2) 

46 INST1 ENTA * 

47 JMP CYCLE Move to the address of the jump. 
48 LEAVE LDA AREG Restore register A. 

49 LEAVEX JMP * Stop tracing. 

50 AREG CON O Simulated rA contents J 


The following things should be noted about trace routines in general and 
this one in particular. 


1) We have presented only the most interesting part of a trace program, 
the part that retains control while executing another program. For a trace to 
be useful, there must also be a routine for writing out the contents of registers, 
and this has not been included. Such a routine distracts from the more subtle 
features of a trace program, although it certainly is important; the necessary 
modifications are left as an exercise (see exercise 2). 

2) Space is generally more important than time; that is, the program should 
be written to be as short as possible. Then the trace routine will be able to coexist 
with extremely large programs. The running time is consumed by output anyway. 

3) Care was taken to avoid destroying the contents of most registers; in fact, 
the program uses only MIX’s A-register. Neither the comparison indicator nor 
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the overflow toggle are affected by the trace routine. (The less we use, the less 
we need to restore.) 
4) When a jump to location JUMP occurs, it is not necessary to ‘STA AREG’, 
since rA cannot have changed. 
5) After leaving the trace routine, the J-register is not reset properly. Exer- 
cise 1 shows how to remedy this. 
6) The program being traced is subject to only three restrictions: 
a) It must not store anything into the locations used by the trace program. 
b) It must not use the output device on which tracing information is being 
recorded (for example, JBUS would give an improper indication). 
c) It will run at a slower speed while being traced. 


EXERCISES 


1. [22] Modify the trace routine of the text so that it restores register J when leaving. 
(You may assume that register J is not zero.) 


2. [26] Modify the trace routine of the text so that before executing each program 
step it writes the following information on tape unit 0. 


Word 1, (0:2) field: location. 

Word 1, (4:5) field: register J (before execution). 

Word 1, (3:3) field: 2 if comparison is greater, 1 if equal, 0 if less; plus 8 if overflow 
is not on before execution. 

Word 2: instruction. 

Word 3: register A (before execution). 

Words 4-9: registers I1-I6 (before execution). 

Word 10: register X (before execution). 


Words 11-100 of each 100-word tape block should contain nine more ten-word groups, 
in the same format. 


3. [10] The previous exercise suggests having the trace program write its output 
onto tape. Discuss why this would be preferable to printing directly. 


4. [25] What would happen if the trace routine were tracing itself? Specifically, 
consider the behavior if the two instructions ENTX LEAVEX; JMP *+1 were placed just 
before ENTER. 


5. [28] In a manner similar to that used to solve the previous exercise, consider 
the situation in which two copies of the trace routine are placed in different places in 
memory, and each is set up to trace the other. What would happen? 


6. [40] Write a trace routine that is capable of tracing itself, in the sense of exercise 4: 
It should print out the steps of its own program at slower speed, and that program will 
be tracing itself at still slower speed, ad infinitum, until memory capacity is exceeded. 


7. [25] Discuss how to write an efficient jump trace routine, which emits much less 
output than a normal trace. Instead of displaying the register contents, a jump trace 
simply records the jumps that occur. It outputs a sequence of pairs (#1, y1), (£2, y2), 

, meaning that the program jumped from location x; to yı, then (after performing 
the instructions in locations yi, yı + 1, ..., v2) it jumped from 22 to y2, etc. [From 
this information it is possible for a subsequent routine to reconstruct the flow of the 
program and to deduce how frequently each instruction was performed.] 
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1.4.4. Input and Output 


Perhaps the most outstanding differences between one computer and the next are 
the facilities available for doing input and output, and the computer instructions 
that govern those peripheral devices. We cannot hope to discuss in a single 
book all of the problems and techniques that arise in this area, so we will 
confine ourselves to a study of typical input-output methods that apply to most 
computers. The input-output operators of MIX represent a compromise between 
the widely varying facilities available in actual machines; to give an example of 
how to think about input-output, let us discuss in this section the problem of 
getting the best MIX input-output. 


D Once again the reader is asked to be indulgent about the anachronistic 
1 MIX computer with its punched cards, etc. Although such old-fashioned 

devices are now quite obsolete, they still can teach important lessons. The MMIX 

computer, when it comes, will of course teach those lessons even better. 


Many computer users feel that input and output are not actually part of 
“real” programming; input and output are considered to be tedious tasks that 
people must perform only because they need to get information in and out of 
a machine. For this reason, the input and output facilities of a computer are 
usually not learned until after all other features have been examined, and it 
frequently happens that only a small fraction of the programmers of a particular 
machine ever know much about the details of input and output. This attitude 
is somewhat natural, because the input-output facilities of machines have never 
been especially pretty. However, the situation cannot be expected to improve 
until more people give serious thought to the subject. We shall see in this section 
and elsewhere (for example, in Section 5.4.6) that some very interesting issues 
arise in connection with input-output, and some pleasant algorithms do exist. 

A brief digression about terminology is perhaps appropriate here. Although 
dictionaries of English formerly listed the words “input” and “output” only as 
nouns (“What kind of input are we getting?”), it is now customary to use them 
grammatically as adjectives (“Don’t drop the input tape.”) and as transitive 
verbs (“Why did the program output this garbage?”). The combined term 
“input-output” is most frequently referred to by the abbreviation “I/O”. In- 
putting is often called reading, and outputting is, similarly, called writing. The 
stuff that is input or output is generally known as “data” — this word is, strictly 
speaking, a plural form of the word “datum,” but it is used collectively as if it 
were singular (“The data has not been read.”), just as the word “information” 
is both singular and plural. This completes today’s English lesson. 

Suppose now that we wish to read from magnetic tape. The IN operator 
of MIX, as defined in Section 1.3.1, merely initiates the input process, and the 
computer continues to execute further instructions while the input is taking 
place. Thus the instruction ‘IN 1000(5)’ will begin to read 100 words from tape 
unit number 5 into memory cells 1000-1099, but the ensuing program must not 
refer to these memory cells until later. The program can assume that input is 
complete only after (a) another I/O operation (IN, OUT, or IOC) referring to unit 5 
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has been initiated, or (b) a conditional jump instruction JBUS(5) or JRED(5) 
indicates that unit 5 is no longer “busy.” 

The simplest way to read a tape block into locations 1000-1099 and to have 
the information present is therefore the sequence of two instructions 


IN 1000(5); JBUS *(5). (1) 


We have used this rudimentary method in the program of Section 1.4.2 (see lines 
07-08 and 52-53). The method is generally wasteful of computer time, however, 
because a very large amount of potentially useful calculating time, say 1000u or 
even 10000u, is consumed by repeated execution of the ‘JBUS’ instruction. The 
program’s running speed can be as much as doubled if this additional time is 
utilized for calculation. (See exercises 4 and 5.) 

One way to avoid such a “busy wait” is to use two areas of memory for the 
input: We can read into one area while computing with the data in the other. 
For example, we could begin our program with the instruction 


IN 2000(5) Begin reading first block. (2) 


Subsequently, we may give the following five commands whenever a tape block 
is desired: 


ENT1 1000 Prepare for MOVE operator. 
JBUS *(5) Wait until unit 5 is ready. 
MOVE 2000(50) (2000-2049) — (1000-1049). (3) 


MOVE 2050(50) (2050-2099) — (1050-1099). 
IN 2000(5) Begin reading next block. 


This has the same overall effect as (1), but it keeps the input tape busy while 
the program works on the data in locations 1000-1099. 

The last instruction of (3) begins to read a tape block into locations 2000- 
2099 before the preceding block has been examined. This is called “reading 
ahead” or anticipated input— it is done on faith that the block will eventually be 
needed. In fact, however, we might discover that no more input is really required, 
after we begin to examine the block in 1000-1099. For example, consider the 
analogous situation in the coroutine program of Section 1.4.2, where the input 
was coming from punched cards instead of tape: A ‘.’ appearing anywhere in the 
card meant that it was the final card of the deck. Such a situation would make 
anticipated input impossible, unless we could assume that either (a) a blank 
card or special trailer card of some other sort would follow the input deck, or 
(b) an identifying mark (e.g., ‘.’) would appear in, say, column 80 of the final 
card of the deck. Some means for terminating the input properly at the end of 
the program must always be provided whenever input has been anticipated. 

The technique of overlapping computation time and I/O time is known as 
buffering, while the rudimentary method (1) is called unbuffered input. The 
area of memory 2000-2099 used to hold the anticipated input in (3), as well as 
the area 1000-1099 to which the input was moved, is called a buffer. Webster’s 
New World Dictionary defines “buffer” as “any person or thing that serves to 
lessen shock,” and the term is appropriate because buffering tends to keep I/O 
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devices running smoothly. (Computer engineers often use the word “buffer” in 
another sense, to denote a part of the I/O device that stores information during 
the transmission. In this book, however, “buffer” will signify an area of memory 
used by a programmer to hold I/O data.) 

The sequence (3) is not always superior to (1), although the exceptions are 
rare. Let us compare the execution times: Suppose T is the time required to 
input 100 words, and suppose C is the computation time that intervenes between 
input requests. Method (1) requires a time of essentially T + C per tape block, 
while method (3) takes essentially max(C, T) + 202u. (The quantity 202u is the 
time required by the two MOVE instructions.) One way to look at this running 
time is to consider “critical path time” —in this case, the amount of time the 
I/O unit is idle between uses. Method (1) keeps the unit idle for C units of time, 
while method (3) keeps it idle for 202 units (assuming that C < T). 

The relatively slow MOVE commands of (3) are undesirable, particularly 
because they take up critical path time when the tape unit must be inactive. 
An almost obvious improvement of the method allows us to avoid these MOVE 
instructions: The outside program can be revised so that it refers alternately 
to locations 1000-1099 and 2000-2099. While we are reading into one buffer 
area, we can be computing with the information in the other; then we can begin 
reading into the second buffer while computing with the information in the first. 
This is the important technique known as buffer swapping. The location of the 
current buffer of interest will be kept in an index register (or, if no index registers 
are available, in a memory location). We have already seen an example of buffer 
swapping applied to output in Algorithm 1.3.2P (see steps P9-P11) and the 
accompanying program. 

As an example of buffer swapping on input, suppose that we have a computer 
application in which each tape block consists of 100 separate one-word items. The 
following program is a subroutine that gets the next word of input and begins 
to read in a new block if the current one is exhausted. 


01 WORDIN STJ 1F Store the exit location. 

02 INC6 1 Advance to the next word. 
03 2H LDA 0,6 Is it the end of the 

04 CMPA =SENTINEL= buffer? 

05 1H JNE * If not, exit. 

06 IN -100,6(U) Refill this buffer. 

O7 LD6 1,6 Switch to the other 

08 JMP 2B buffer and return. (4) 
09 INBUF1 ORIG *+100 First buffer 

10 CON SENTINEL Sentinel at end of buffer 
11 CON *+1 Address of other buffer 

12 INBUF2 ORIG *+100 Second buffer 

13 CON SENTINEL Sentinel at end of buffer 
14 CON INBUF1 Address of other buffer | 


In this routine, index register 6 is used to address the last word of input; we 
assume that the calling program does not affect this register. The symbol U 
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refers to a tape unit, and the symbol SENTINEL refers to a value that is known 
(from characteristics of the program) to be absent from all tape blocks. 
Several things about this subroutine should be noted: 


1) The sentinel constant appears as the 101st word of each buffer, and it 
makes a convenient test for the end of the buffer. In many applications, however, 
the sentinel technique will not be reliable, since any word may appear on tape. 
If we were doing card input, a similar method (with the 17th word of the buffer 
equal to a sentinel) could always be used without fear of failure; in that case, 
any negative word could serve as a sentinel, since MIX input from cards always 
gives nonnegative words. 

2) Each buffer contains the address of the other buffer (see lines 07, 11, 
and 14). This “linking together” facilitates the swapping process. 

3) No JBUS instruction was necessary, since the next input was initiated 
before any word of the previous block was accessed. If the quantities C and T 
refer as before to computation time and tape time, the execution time per tape 
block is now max (C,T); it is therefore possible to keep the tape going at full 
speed if C < T. (Note: MIX is an idealized computer in this regard, however, 
since no I/O errors must be treated by the program. On most machines some 
instructions to test the successful completion of the previous operation would be 
necessary just before the ‘IN’ instruction here.) 

4) To make subroutine (4) work properly, it will be necessary to get things 
started out right when the program begins. Details are left to the reader (see 
exercise 6). 

5) The WORDIN subroutine makes the tape unit appear to have a block length 
of 1 rather than 100 as far as the rest of the program is concerned. The idea 
of having several program-oriented records filling a single actual tape block is 
called blocking of records. 

The techniques that we have illustrated for input apply, with minor changes, 
to output as well (see exercises 2 and 3). 


NEXTR 


Fig. 23. A circle of buffers (N = 6). 


Multiple buffers. Buffer swapping is just the special case N = 2 of a general 
method involving N buffers. In some applications it is desirable to have more 
than two buffers; for example, consider the following type of algorithm: 
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Step 1. Read five blocks in rapid succession. 
Step 2. Perform a fairly long calculation based on this data. 
Step 3. Return to step 1. 


Here five or six buffers would be desirable, so that the next batch of five blocks 
could be read during step 2. This tendency for I/O activity to be “bunched” 
makes multiple buffering an improvement over buffer swapping. 

Suppose we have N buffers for some input or output process using a single 
I/O device; we will imagine that the buffers are arranged in a circle, as in 
Fig. 23. The program external to the buffering process can be assumed to have 
the following general form with respect to the I/O unit of interest: 


ASSIGN 
RELEASE 
ASSIGN 


RELEASE 


in other words, we can assume that the program alternates between an action 
called “ASSIGN” and an action called “RELEASE”, separated by other computa- 
tions that do not affect the allocation of buffers. 


ASSIGN means that the program acquires the address of the next buffer area; 
this address is assigned as the value of some program variable. 


RELEASE means that the program is done with the current buffer area. 


Between ASSIGN and RELEASE the program is communicating with one of the 
buffers, called the current buffer area; between RELEASE and ASSIGN, the program 
makes no reference to any buffer area. 

Conceivably, ASSIGN could immediately follow RELEASE, and discussions of 
buffering have often been based on this assumption. However, if RELEASE is done 
as soon as possible, the buffering process has more freedom and will be more 
effective; by separating the two essentially different functions of ASSIGN and 
RELEASE we will find that the buffering technique remains easy to understand, 
and our discussion will be meaningful even if N = 1. 

To be more explicit, let us consider the cases of input and output separately. 
For input, suppose we are dealing with a card reader. The action ASSIGN means 
that the program needs to see information from a new card; we would like to set 
an index register to the memory address at which the next card image is located. 
The action RELEASE occurs when the information in the current card image is no 
longer needed — it has somehow been digested by the program, perhaps copied 
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to another part of memory, etc. The current buffer area may therefore be filled 
with further anticipated input. 

For output, consider the case of a line printer. The action ASSIGN occurs 
when a free buffer area is needed, into which a line image is to be placed for 
printing. We wish to set an index register equal to the memory address of such 
an area. The action RELEASE occurs when this line image has been fully set up 
in the buffer area, in a form ready to be printed. 


Example: To print the contents of locations 0800-0823, we might write 


JMP ASSIGNP (Sets rI5 to buffer location) 
ENT1 0,5 (5) 
MOVE 800(24) Move 24 words into the output buffer. 
JMP RELEASEP 
where ASSIGNP and RELEASEP represent subroutines to do the two buffering 
functions for the line printer. 


In an optimal situation, from the standpoint of the computer, the ASSIGN 
operation will require virtually no execution time. This means, on input, that 
each card image will have been anticipated, so that the data is available when 
the program is ready for it; and on output, it means that there will always be 
a free place in memory to record the line image. In either case, no time will be 
spent waiting for the I/O device. 

To help describe the buffering algorithm, and to make it more colorful, we 
will say that buffer areas are either green, yellow, or red (shown as G, Y, and R 
in Fig. 24). 

Green means that the area is ready to be ASSIGNed; this means that it has 
been filled with anticipated information (in an input situation), or that it is a 
free area (in an output situation). 

Yellow means that the area has been ASSIGNed, not RELEASEd; this means 
that it is the current buffer, and the program is communicating with it. 

Red means that the area has been RELEASEd; thus it is a free area (in an 
input situation) or it has been filled with information (in an output situation). 

Figure 23 shows two “pointers” associated with the circle of buffers. These 
are, conceptually, index registers in the program. NEXTG and NEXTR point to 
the “next green” and “next red” buffer, respectively. A third pointer, CURRENT 
(shown in Fig. 24), indicates the yellow buffer when one is present. 


The algorithms below apply equally well to input or output, but for defi- 
niteness we will consider first the case of input from a card reader. Suppose that 
a program has reached the state shown in Fig. 23. This means that four card 
images have been anticipated by the buffering process, and they reside in the 
green buffers. At this moment, two things are happening simultaneously: (a) The 
program is computing, following a RELEASE operation; (b) a card is being read 
into the buffer indicated by NEXTR. This state of affairs will continue until the 
input cycle is completed (the unit will then go from “busy” to “ready”), or until 
the program does an ASSIGN operation. Suppose the latter occurs first; then the 
buffer indicated by NEXTG changes to yellow (it is assigned as the current buffer), 
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NEXTG NEXTG NEXTG 


Fig. 24. Buffer transitions, (a) after ASSIGN, (b) after I/O complete, and (c) after 
RELEASE. 


NEXTG moves clockwise, and we arrive at the position shown in Fig. 24(a). If 
now the input is completed, another anticipated block is present; so the buffer 
changes from red to green, and NEXTR moves over as shown in Fig. 24(b). If the 
RELEASE operation follows next, we obtain Fig. 24(c). 

For an example concerning output, see Fig. 27 on page 226. That illustration 
shows the “colors” of buffer areas as a function of time, in a program that opens 
with four quick outputs, then produces four at a slow pace, and finally issues two 
in rapid succession as the program ends. Three buffers appear in that example. 

The pointers NEXTR and NEXTG proceed merrily around the circle, each at an 
independent rate of speed, moving clockwise. It is a race between the program 
(which turns buffers from green to red) and the I/O buffering process (which 
turns them from red to green). Two situations of conflict can occur: 


a) if NEXTG tries to pass NEXTR, the program has gotten ahead of the I/O device 
and it must wait until the device is ready. 

b) if NEXTR tries to pass NEXTG, the I/O device has gotten ahead of the program 
and we must shut it down until the next RELEASE is given. 


Both of these situations are depicted in Fig. 24. (See exercise 9.) 

Fortunately, in spite of the rather lengthy explanation just given of the ideas 
behind a circle of buffers, the actual algorithms for handling the situation are 
quite simple. In the following description, 


N = total number of buffers; (6) 
n = current number of red buffers. 

The variable n is used in the algorithm below to avoid interference between 
NEXTG and NEXTR. 


Algorithm A (ASSIGN). This algorithm includes the steps implied by ASSIGN 

within a computational program, as described above. 

Al. [Wait for n < N] If n = N, stall the program until n < N. (If n = N, 
no buffers are ready to be assigned; but Algorithm B below, which runs in 
parallel with this one, will eventually succeed in producing a green buffer.) 
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A2. [CURRENT <+ NEXTG.] Set CURRENT « NEXTG (thereby assigning the current 
buffer). 


A3. [Advance NEXTG.] Advance NEXTG to the next clockwise buffer. | 
Algorithm R (RELEASE). This algorithm includes the steps implied by RELEASE 
within a computational program, as described above. 

R1. [Increase n.] Increase n by one. J 

Algorithm B (Buffer control). This algorithm performs the actual initiation 


of I/O operators in the machine; it is to be executed “simultaneously” with the 
main program, in the sense described below. 
B1. [Compute.] Let the main program compute for a short period of time; step 


B2 will be executed after a certain time delay, at a time when the I/O device 
is ready for another operation. 


B2. [n = 0?] If n = 0, go to B1. (Thus, if no buffers are red, no I/O action can 
be performed.) 


B3. [Initiate I/O.] Initiate transmission between the buffer area designated by 
NEXTR and the I/O device. 


B4. [Compute.] Let the main program run for a period of time; then go to 
step B5 when the I/O operation is completed. 


B5. [Advance NEXTR.] Advance NEXTR to the next clockwise buffer. 


B6. [Decrease n.] Decrease n by one, and go to B2. I 


In these algorithms, we have two independent processes going on “simultane- 
ously,” the buffering control program and the computation program. These pro- 
cesses are, in fact, coroutines, which we will call CONTROL and COMPUTE. Coroutine 
CONTROL jumps to COMPUTE in steps B1 and B4; coroutine COMPUTE jumps to 
CONTROL by interspersing “jump ready” instructions at sporadic intervals in its 
program. 

Coding this algorithm for MIX is extremely simple. For convenience, assume 
that the buffers are linked so that the word preceding each one is the address of 
the next; for example, with N = 3 buffers we have CONTENTS (BUF1 — 1) = BUF2, 
CONTENTS (BUF2 — 1) = BUF3, and CONTENTS (BUF3 — 1) = BUF1. 


Program A (ASSIGN, a subroutine within the COMPUTE coroutine). rl4 = 
CURRENT; rI6 = n; calling sequence is JMP ASSIGN; on exit, rX contains NEXTG. 


ASSIGN STJ 9F Subroutine linkage 

1H JRED CONTROL(U) A1. Wait forn < N. 
CMP6 =N= 
JE 1B 
LD4 NEXTG A2. CURRENT 4+ NEXTG. 
LDX -1,4 A3. Advance NEXTG. 
STX NEXTG 


9H JMP * Exit. J 
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ASSIGN RELEASE 
Al. ` A2. x A3. R1. 
Wait for n< N CURRENT <— NEXTG Advance NEXTG Increase n 


BUFFER CONTROL 


B5. TS 
B3. s 4 B4. À 5 ` B6. 
Initiate I/O Compute Adone Decrease n 


Fig. 25. Algorithms for multiple buffering. 


Program R (RELEASE, code used within the COMPUTE coroutine). r16 =n. This 
short code is to be inserted wherever RELEASE is desired. 


INC6 1 R1. Increase n. 
JRED CONTROL(U) Possible jump to CONTROL coroutine | 


Program B (The CONTROL coroutine). rI6 = n, rI5 = NEXTR. 
CONT1 JMP COMPUTE B1. Compute. 
1H J6Z *-1 B2. n= 0? 


IN 0,5(U) B3. Initiate I/O. 
JMP COMPUTE B4. Compute. 


LD5 -1,5 B5. Advance NEXTR. 
DEC6 1 B6. Decrease n. 
JMP 1B I 


Besides the code above, we also have the usual coroutine linkage 


CONTROL STJ COMPUTEX COMPUTE STJ CONTROLX 
CONTROLX JMP CONT1 COMPUTEX JMP COMP1 


and the instruction ‘JRED CONTROL (U)’ should be placed within COMPUTE about 
once in every fifty instructions. 

Thus the programs for multiple buffering essentially amount to only seven 
instructions for CONTROL, eight for ASSIGN, and two for RELEASE. 

It is perhaps remarkable that exactly the same algorithm will work for both 
input and output. What is the difference— how does the control routine know 
whether to anticipate (for input) or to lag behind (for output)? The answer lies 
in the initial conditions: For input we start out with n = N (all buffers red) and 
for output we start out with n = 0 (all buffers green). Once the routine has been 
started properly, it continues to behave as either an input process or an output 
process, respectively. The other initial condition is that NEXTR = NEXTG, both 
pointing at one of the buffers. 

At the conclusion of the program, it is necessary to stop the I/O process (if 
it is input) or to wait until it is completed (for output); details are left to the 
reader (see exercises 12 and 13). 
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It is important to ask what is the best value of N to use. Certainly as N 
gets larger, the speed of the program will not decrease, but it will not increase 
indefinitely either and so we come to a point of diminishing returns. Let us refer 
again to the quantities C and T, representing computation time between I/O 
operators and the I/O time itself. More precisely, let C be the amount of time 
between successive ASSIGNs, and let T be the amount of time needed to transmit 
one block. If C is always greater than T, then N = 2 is adequate, for it is not 
hard to see that with two buffers we keep the computer busy at all times. If 
C is always less than T, then again N = 2 is adequate, for we keep the I/O 
device busy at all times (except when the device has special timing constraints 
as in exercise 19). Larger values of N are therefore useful chiefly when C varies 
between small values and large values; the average number of consecutive small 
values, plus 1, may be right for N, if the large values of C are significantly longer 
than T. (However, the advantage of buffering is virtually nullified if all input 
occurs at the beginning of the program and if all output occurs at the end.) 
If the time between ASSIGN and RELEASE is always quite small, the value of N 
may be decreased by 1 throughout the discussion above, with little effect on 
running time. 

This approach to buffering can be adapted in many ways, and we will 
mention a few of them briefly. So far we have assumed that only one I/O device 
was being used; in practice, of course, several devices will be in use at the 
same time. 

There are several ways to approach the subject of multiple units. In the 
simplest case, we can have a separate circle of buffers for each device. Each 
unit will have its own values of n, N, NEXTR, NEXTG, and CURRENT, and its own 
CONTROL coroutine. This will give efficient buffering action simultaneously on 
every I/O device. 

It is also possible to “pool” buffer areas that are of the same size, so that two 
or more devices share buffers from a common list. This can be handled by using 
the linked memory techniques of Chapter 2, with all red input buffers linked 
together in one list and all green output buffers linked together in another. It 
becomes necessary to distinguish between input and output in this case, and 
to rewrite the algorithms without using n and N. The algorithm may get 
irrevocably stuck, if all buffers in the pool are filled with anticipated input; 
so a check should be made that there is always at least one buffer (preferably 
one for each device) that is not input-green; only if the COMPUTE routine is stalled 
at step Al for some input device should we allow input into the final buffer of 
the pool from that device. 

Some machines have additional constraints on the use of input-output units, 
so that it is impossible to be transmitting data from certain pairs of devices at 
the same time. (For example, several units might be attached to the computer by 
means of a single “channel.”) This constraint also affects our buffering routine; 
when we must choose which I/O unit to initiate next, how is the choice to be 
made? This is called the problem of “forecasting.” The best forecasting rule for 
the general case would seem to give preference to the unit whose buffer circle 
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NEXTR 


Fig. 26. Input and output from the same circle. 


NEXTG 


has the largest value of n/N, assuming that the number of buffers in the circles 
has been chosen wisely. 

Let’s conclude this discussion by taking note of a useful method for doing 
both input and output from the same buffer circle, under certain conditions. 
Figure 26 introduces a new kind of buffer, which has the color purple. In this 
situation, green buffers represent anticipated input; the program ASSIGNs and 
a green buffer becomes yellow, then upon RELEASE it turns red and represents 
a block to be output. The input and output processes follow around the circle 
independently as before, except that now we turn red buffers to purple after 
the output is done, and convert purple to green on input. It is necessary to 
ensure that none of the pointers NEXTG, NEXTR, NEXTP pass each other. At 
the instant shown in Fig. 26, the program is computing between ASSIGN and 
RELEASE, while accessing the yellow buffer; simultaneously, input is going into 
the buffer indicated by NEXTP; and output is coming from the buffer indicated 
by NEXTR. 


EXERCISES 


1. [05] (a) Would sequence (3) still be correct if the MOVE instructions were placed 
before the JBUS instruction instead of after it? (b) What if the MOVE instructions were 
placed after the IN command? 


2. [10] The instructions ‘OUT 1000(6); JBUS *(6)’ may be used to output a tape 
block in an unbuffered fashion, just as the instructions (1) did this for input. Give a 
method analogous to (2) and (3) that buffers this output, by using MOVE instructions 
and an auxiliary buffer in locations 2000-2099. 


3. [22] Write a buffer-swapping output subroutine analogous to (4). The subroutine, 
called WORDOUT, should store the word in rA as the next word of output, and if a buffer 
is full it should write 100 words onto tape unit V. Index register 5 should be used to 
refer to the current buffer position. Show the layout of buffer areas and explain what 
instructions (if any) are necessary at the beginning and end of the program to ensure 
that the first and last blocks are properly written. The final block should be filled out 
with zeros if necessary. 


4. [M20] Show that if a program refers to a single I/O device, we might be able to 
cut the running time in half by buffering the I/O, in favorable circumstances; but we 
can never decrease the running time by more than a factor of two, with respect to the 
time taken by unbuffered I/O. 
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Fig. 27. Output with three buffers (see exercise 9). 


> 5. [M21] Generalize the situation of the preceding exercise to the case when the 
program refers to n I/O devices instead of just one. 
6. [12] What instructions should be placed at the beginning of a program so that 
the WORDIN subroutine (4) gets off to the right start? (For example, index register 6 
must be set to something.) 


7. [22] Write a subroutine called WORDIN that is essentially like (4) except that it 
does not make use of a sentinel. 


8. [11] The text describes a hypothetical input scenario that leads from Fig. 23 
through parts (a), (b), and (c) of Fig. 24. Interpret the same scenario under the 
assumption that output to the line printer is being done, instead of input from cards. 
(For example, what things are happening at the time shown in Fig. 237) 


> 9. [21] A program that leads to the buffer contents shown in Fig. 27 may be char- 
acterized by the following list of times: 


A, 1000, R, 1000, A, 1000, R, 1000, A, 1000, R, 1000, A, 1000, R, 1000, 
A, 7000, R, 5000, A, 7000, R, 5000, A, 7000, R, 5000, A, 7000, R, 5000, 
A, 1000, R, 1000, A, 2000, R, 1000. 


This list means “assign, compute for 1000u, release, compute for 1000u, assign, ..., 
compute for 2000u, release, compute for 1000u.” The computation times given do not 
include any intervals during which the computer might have to wait for the output 
device to catch up (as at the fourth “assign” in Fig. 27). The output device operates 
at a speed of 7500u per block. 
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The following chart specifies the actions shown in Fig. 27 as time passes: 


Time Action Time Action 
Q ASSIGN (BUF1) 38500 OUT BUF3 
1000 RELEASE, OUT BUF1 40000 ASSIGN(BUF1) 
2000 ASSIGN (BUF2) 46000 Output stops. 
3000 RELEASE 47000 RELEASE, OUT BUF1 
4000 ASSIGN(BUF3) 52000 ASSIGN(BUF2) 
5000 RELEASE 54500 Output stops. 
6000 ASSIGN (wait) 59000 RELEASE, OUT BUF2 
8500 BUF1 assigned, OUT BUF2 64000 ASSIGN(BUF3) 
9500 RELEASE 65000 RELEASE 
10500 ASSIGN (wait) 66000 ASSIGN(BUF1) 
16000 BUF2 assigned, OUT BUF3 66500 OUT BUF3 
23000 RELEASE 68000 RELEASE 
23500 OUT BUF1 69000 Computation stops. 
28000 ASSIGN (BUF3) 74000 OUT BUF1 
31000 OUT BUF2 81500 Output stops. 


35000 RELEASE 


The total time required was therefore 81500u; the computer was idle from 6000-8500, 
10500-16000, and 69000-81500, or 20500u altogether; the output unit was idle from 
0-1000, 46000-47000, and 54500-59000, or 6500u. 

Make a time-action chart like the above for the same program, assuming that there 
are only two buffers. 


10. [21] Repeat exercise 9, except with four buffers. 
11. [21] Repeat exercise 9, except with just one buffer. 


12. [24] Suppose that the multiple buffering algorithm in the text is being used for 
card input, and suppose the input is to terminate as soon as a card with “.” in column 
80 has been read. Show how the CONTROL coroutine (Algorithm B and Program B) 
should be changed so that input is shut off in this way. 


13. [20] What instructions should be included at the end of the COMPUTE coroutine 
in the text, if the buffering algorithms are being applied to output, to ensure that all 
information has been output from the buffers? 


14. [20] Suppose the computational program does not alternate between ASSIGN and 
RELEASE, but instead gives the sequence of actions ...ASSIGN ...ASSIGN ...RELEASE... 
RELEASE. What effect does this have on the algorithms described in the text? Is it 
possibly useful? 


15. [22] Write a complete MIX program that copies 100 blocks from tape unit 0 to 
tape unit 1, using just three buffers. The program should be as fast as possible. 


16. [29] Formulate the “green-yellow-red-purple” algorithm, suggested by Fig. 26, 
in the manner of the algorithms for multiple buffering given in the text, using three 
coroutines (one to control the input device, one for the output device, and one for the 
computation). 


17. [40] Adapt the multiple-buffer algorithm to pooled buffers; build in methods that 
keep the process from slowing down, due to too much anticipated input. Try to make 
the algorithm as elegant as possible. Compare your method to nonpooling methods, 
applied to real-life problems. 
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> 18. [30] A proposed extension of MIX allows its computations to be interrupted, as 
explained below. Your task in this exercise is to modify Algorithms and Programs A, 
R, and B of the text so that they use these interrupt facilities instead of the ‘JRED’ 
instructions. 

The new MIX features include an additional 3999 memory cells, locations —3999 
through —0001. The machine has two internal “states,” normal state and control state. 
In normal state, locations —3999 through —0001 are not admissible memory locations 
and the MIX computer behaves as usual. When an “interrupt” occurs, due to conditions 
explained later, locations —0009 through —0001 are set equal to the contents of MIX’s 
registers: rA in —0009; rI1 through rI6 in —0008 through —0003; rX in —0002; and rJ, 
the overflow toggle, the comparison indicator, and the location of the next instruction 
are stored in —0001 as 


next jov,) o |. 
+| inst. [er | T |} 


the machine enters control state, at a location depending on the type of interrupt. 

Location —0010 acts as a “clock”: Every 1000u of time, the number appearing 
in this location is decreased by one, and if the result is zero an interrupt to location 
—0011 occurs. 

The new MIX instruction ‘INT’ (C = 5, F = 9) works as follows: (a) In normal state, 
an interrupt occurs to location —0012. (Thus a programmer may force an interrupt, 
to communicate with a control routine; the address of INT has no effect, although the 
control routine may use it for information to distinguish between types of interrupt.) 
(b) In control state, all MIX registers are loaded from locations —0009 to —0001, the 
computer goes into normal state, and it resumes execution. The execution time for INT 
is 2u in each case. 

An IN, OUT, or IOC instruction given in control state will cause an interrupt to occur 
as soon as the I/O operation is completed. The interrupt goes to location —(0020+ 
unit number). 

No interrupts occur while in control state; any interrupt conditions are “saved” 
until after the next INT operation, and interrupt will occur after one instruction of the 
normal state program has been performed. 


> 19. [M28] Special considerations arise when input or output involves short blocks on 
a rotating device like a magnetic disk. Suppose a program works with n > 2 consecutive 
blocks of information in the following way: Block k begins to be input at time tk, where 
tı = 0. It is assigned for processing at time up > tk +T and released from its buffer at 
time Vk = uk + C. The disk rotates once every P units of time, and its reading head 
passes the start of a new block every L units; so we must have tp = (k—1)L (modulo P). 
Since the processing is sequential, we must also have up > vg_i for 1 < k < n. There 
are N buffers, hence tk > Uk-yn for N<k <n. 

How large does N have to be so that the finishing time vn has its minimum possible 
value, T + C +(n—1)max(L,C)? Give a general rule for determining the smallest 
such N. Illustrate your rule when L = 1, P = 100, T = .5, n = 100, and (a) C = .5; 
(b) C = 1.0; (c) C = 1.01; (d) C = 1.5; (e) C = 2.0; (f) C = 2.5; (g) C = 10.0; 
(h) C = 50.0; (i) C = 200.0. 
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1.4.5. History and Bibliography 

Most of the fundamental techniques described in Section 1.4 have been developed 
independently by a number of different people, and the exact history of the ideas 
will probably never be known. An attempt has been made to record here the 
most important contributions to the history, and to put them in perspective. 

Subroutines were the first labor-saving devices invented for programmers. 
In the 19th century, Charles Babbage envisioned a library of routines for his 
Analytical Engine [see Charles Babbage and His Calculating Engines, edited by 
Philip and Emily Morrison (Dover, 1961), 56]; and we might say that his dream 
came true in 1944 when Grace M. Hopper wrote a subroutine for computing 
sin x on the Harvard Mark I calculator [see Mechanisation of Thought Processes 
(London: Nat. Phys. Lab., 1959), 164]. However, these were essentially “open 
subroutines,” meant to be inserted into a program where needed instead of being 
linked up dynamically. Babbage’s planned machine was controlled by sequences 
of punched cards, as on the Jacquard loom; the Mark I was controlled by a 
number of paper tapes. Thus they were quite different from today’s stored- 
program computers. 

Subroutine linkage appropriate to stored-program machines, with the return 
address supplied as a parameter, was discussed by Herman H. Goldstine and 
John von Neumann in their widely circulated monograph on programming, 
written during 1946 and 1947; see von Neumann’s Collected Works 5 (New York: 
Macmillan, 1963), 215-235. The main routine of their programs was responsible 
for storing parameters into the body of the subroutine, instead of passing the 
necessary information in registers. In England, A. M. Turing had designed 
hardware and software for subroutine linkage as early as 1945; see Proceedings of 
a Second Symposium on Large-Scale Digital Calculating Machinery (Cambridge, 
Mass.: Harvard University, 1949), 87-90; B. E. Carpenter and R. W. Doran, 
editors, A. M. Turing’s ACE Report of 1946 and Other Papers (Cambridge, 
Mass.: MIT Press, 1986), 35-36, 76, 78-79. The use and construction of a 
very versatile subroutine library is the principal topic of the first textbook of 
computer programming, The Preparation of Programs for an Electronic Digital 
Computer, by M. V. Wilkes, D. J. Wheeler, and S. Gill, 1st ed. (Reading, Mass.: 
Addison—Wesley, 1951). 

The word “coroutine” was coined by M. E. Conway in 1958, after he had 
developed the concept, and he first applied it to the construction of an assembly 
program. Coroutines were independently studied by J. Erdwinn and J. Merner, 
at about the same time; they wrote a paper entitled “Bilateral Linkage,” which 
was not then considered sufficiently interesting to merit publication, and un- 
fortunately no copies of that paper seem to exist today. The first published 
explanation of the coroutine concept appeared much later in Conway’s article 
“Design of a Separable Transition-Diagram Compiler,” CACM 6 (1963), 396- 
408. Actually a primitive form of coroutine linkage had already been noted briefly 
as a “programming tip” in an early UNIVAC publication [The Programmer 1,2 
(February 1954), 4]. A suitable notation for coroutines in ALGOL-like languages 
was introduced in Dahl and Nygaard’s SIMULA I [CACM 9 (1966), 671-678], 
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and several excellent examples of coroutines (including replicated coroutines) 
appear in the book Structured Programming by O.-J. Dahl, E. W. Dijkstra, and 
C. A. R. Hoare, Chapter 3. 

The first interpretive routine may be said to be the “Universal Turing 
Machine,” a Turing machine capable of simulating any other Turing machines. 
Turing machines are not actual computers; they are theoretical constructions 
used to prove that certain problems are unsolvable by algorithms. Interpretive 
routines in the conventional sense were mentioned by John Mauchly in his 
lectures at the Moore School in 1946. The most notable early interpreters, 
chiefly intended to provide a convenient means of doing floating point arithmetic, 
were certain routines for the Whirlwind I (by C. W. Adams and others) and 
for the ILLIAC I (by D. J. Wheeler and others). Turing took a part in this 
development also; interpretive systems for the Pilot ACE computer were written 
under his direction. For references to the state of interpreters in the early fifties, 
see the article “Interpretative Sub-routines” by J. M. Bennett, D. G. Prinz, 
and M. L. Woods, Proc. ACM (Toronto: 1952), 81-87; see also various papers 
in the Proceedings of the Symposium on Automatic Programming for Digital 
Computers (1954), published by the Office of Naval Research, Washington, D.C. 

The most extensively used early interpretive routine was probably John 
Backus’s “IBM 701 Speedcoding system” [see JACM 1 (1954), 4-6]. This 
interpreter was slightly modified and skillfully rewritten for the IBM 650 by 
V. M. Wolontis and others of the Bell Telephone Laboratories; their routine, 
called the “Bell Interpretive System,” became extremely popular. The IPL 
interpretive systems, which were designed beginning in 1956 by A. Newell, J. C. 
Shaw, and H. A. Simon for applications to quite different problems (see Section 
2.6), were used extensively for list processing. Modern uses of interpreters, as 
mentioned in the introduction to Section 1.4.3, are often mentioned in passing 
in the computer literature; see the references listed in that section for articles 
that discuss interpreters in somewhat more detail. 

The first tracing routine was developed by Stanley Gill in 1950; see his 
interesting article in Proceedings of the Royal Society of London, series A, 
206 (1951), 538-554. The text by Wilkes, Wheeler, and Gill mentioned above 
includes several programs for tracing. Perhaps the most interesting of them is 
subroutine C-10 by D. J. Wheeler, which includes a provision for suppressing 
the trace upon entry to a library subroutine, executing the subroutine at full 
speed, then continuing the trace. Published information about trace routines is 
quite rare in the general computer literature, primarily because the methods are 
inherently oriented to a particular machine. The only other early reference known 
to the author is H. V. Meek, “An Experimental Monitoring Routine for the IBM 
705,” Proc. Western Joint Computer Conf. (1956), 68-70, which discusses a trace 
routine for a machine on which the problem was particularly difficult. See also 
the trace routine for IBM’s System/360 architecture, presented in A Compiler 
Generator by W. M. McKeeman, J. J. Horning, and D. B. Wortman (Prentice— 
Hall, 1970), 305-363. Nowadays the emphasis on trace routines has shifted to 
software that provides selective symbolic output and measurements of program 
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performance; one of the best such systems was developed by E. Satterthwaite, 
and described in Software Practice & Experience 2 (1972), 197-217. 

Buffering was originally performed by computer hardware, in a manner 
analogous to the code 1.4.4—(3); an internal buffer area inaccessible to the pro- 
grammer played the role of locations 2000-2099, and the instructions 1.4.4—(3) 
were implicitly performed behind the scenes when an input command was given. 
During the late 1940s, software buffering techniques that are especially useful for 
sorting were developed by early programmers of the UNIVAC (see Section 5.5). 
For a good survey of the prevailing philosophy towards I/O in 1952, see the 
proceedings of the Eastern Joint Computer Conference held in that year. 

The DYSEAC computer [Alan L. Leiner, JACM 1 (1954), 57-81] introduced 
the idea of input-output devices communicating directly with memory while a 
program is running, then interrupting the program upon completion. Such a 
system implies that buffering algorithms had been developed, but the details 
went unpublished. The first published reference to buffering techniques in the 
sense we have described gives a highly sophisticated approach; see O. Mock and 
C. J. Swift, “Programmed Input-Output Buffering,” Proc. ACM Nat. Meeting 
13 (1958), paper 19, and JACM 6 (1959), 145-151. (The reader is cautioned 
that both articles contain a good deal of local jargon, which may take some time 
to understand, but neighboring articles in JACM 6 will help.) An interrupt 
system that enabled buffering of input and output was independently developed 
by E. W. Dijkstra in 1957 and 1958, in connection with B. J. Loopstra’s and 
C. S. Scholten’s X1 computer [see Comp. J. 2 (1959), 39-43]. Dijkstra’s doc- 
toral thesis, “Communication with an Automatic Computer” (1959), discussed 
primitive synchronization operations by which users could create long chains 
of buffers with respect to paper tape and typewriter I/O; each buffer contained 
either a single character or a single number. He later developed the ideas into the 
important general notion of semaphores, which are basic to the control of all sorts 
of concurrent processes, not just input-output [see Programming Languages, 
ed. by F. Genuys (Academic Press, 1968), 43-112; BIT 8 (1968), 174-186; 
Acta Informatica 1 (1971), 115-138]. The paper “Input-Output Buffering and 
FORTRAN” by David E. Ferguson, JACM 7 (1960), 1-9, describes buffer circles 
and gives a detailed description of simple buffering with many units at once. 


About 1,000 instructions is a reasonable upper limit 
for the complexity of problems now envisioned. 


— HERMAN GOLDSTINE and JOHN VON NEUMANN (1946) 


CHAPTER TWO 


INFORMATION STRUCTURES 


| think that | shall never see 
A poem lovely as a tree. 


— JOYCE KILMER (1913) 


Yea, from the table of my memory 
I'll wipe away all trivial fond records. 


— HAMLET (Act I, Scene 5, Line 98) 


2.1. INTRODUCTION 


COMPUTER PROGRAMS usually operate on tables of information. In most cases 
these tables are not simply amorphous masses of numerical values; they involve 
important structural relationships between the data elements. 

In its simplest form, a table might be a linear list of elements, when its 
relevant structural properties might include the answers to such questions as: 
Which element is first in the list? Which is last? Which elements precede and 
follow a given one? How many elements are in the list? A lot can be said about 
structure even in this apparently simple case (see Section 2.2). 

In more complicated situations, the table might be a two-dimensional array 
(a matrix or grid, having both a row and a column structure), or it might be 
an n-dimensional array for higher values of n; it might be a tree structure, 
representing hierarchical or branching relationships; or it might be a complex 
multilinked structure with a great many interconnections, such as we may find 
in a human brain. 

In order to use a computer properly, we need to understand the structural 
relationships present within data, as well as the basic techniques for representing 
and manipulating such structure within a computer. 

The present chapter summarizes the most important facts about information 
structures: the static and dynamic properties of different kinds of structure; 
means for storage allocation and representation of structured data; and efficient 
algorithms for creating, altering, accessing, and destroying structural informa- 
tion. In the course of this study, we will also work out several important examples 
that illustrate the application of such methods to a wide variety of problems. 
The examples include topological sorting, polynomial arithmetic, discrete system 
simulation, sparse matrix transformation, algebraic formula manipulation, and 
applications to the writing of compilers and operating systems. Our concern 
will be almost entirely with structure as represented inside a computer; the 
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conversion from external to internal representations is the subject of Chapters 9 
and 10. 

Much of the material we will discuss is often called “List processing,” since 
a number of programming systems such as LISP have been designed to facilitate 
working with general kinds of structures called Lists. (When the word “list” 
is capitalized in this chapter, it is being used in a technical sense to denote 
a particular type of structure that is highlighted in Section 2.3.5.) Although 
List processing systems are useful in a large number of situations, they impose 
constraints on the programmer that are often unnecessary; it is usually better to 
use the methods of this chapter directly in one’s own programs, tailoring the data 
format and the processing algorithms to the particular application. Many people 
unfortunately still feel that List processing techniques are quite complicated (so 
that it is necessary to use someone else’s carefully written interpretive system or 
a prefabricated set of subroutines), and that List processing must be done only 
in a certain fixed way. We will see that there is nothing magic, mysterious, or 
difficult about the methods for dealing with complex structures; these techniques 
are an important part of every programmer’s repertoire, and we can use them 
easily whether we are writing a program in assembly language or in an algebraic 
language like FORTRAN, C, or Java. 

We will illustrate methods of dealing with information structures in terms 
of the MIX computer. A reader who does not care to look through detailed 
MIX programs should at least study the ways in which structural information is 
represented in MIX’s memory. 

It is important at this point to define several terms and notations that we 
will be using frequently from now on. The information in a table consists of a 
set of nodes (called “records,” “entities,” or “beads” by some authors); we will 
occasionally say “item” or “element” instead of “node.” Each node consists of 
one or more consecutive words of the computer memory, divided into named 
parts called fields. In the simplest case, a node is just one word of memory, and 
it has just one field comprising that whole word. As a more interesting example, 
suppose the elements of our table are intended to represent playing cards; we 
might have two-word nodes broken into five fields, TAG, SUIT, RANK, NEXT, and 
TITLE: 


+ | TAG | SUIT | RANK NEXT 


! (1) 


+ TITLE 


(This format reflects the contents of two MIX words. Recall that a MIX word 
consists of five bytes and a sign; see Section 1.3.1. In this example we assume 
that the signs are + in each word.) The address of a node, also called a link, 
pointer, or reference to that node, is the memory location of its first word. The 
address is often taken relative to some base location, but in this chapter for 
simplicity we will take the address to be an absolute memory location. 

The contents of any field within a node may represent numbers, alphabetic 
characters, links, or anything else the programmer may desire. In connection 
with the example above, we might wish to represent a pile of cards that might 
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appear in a game of solitaire: TAG = 1 means that the card is face down, TAG = 0 
means that it is face up; SUIT = 1, 2, 3, or 4 for clubs, diamonds, hearts, or 
spades, respectively; RANK = 1, 2, ..., 13 for ace, deuce, ..., king; NEXT is a link 
to the card below this one in the pile; and TITLE is a five-character alphabetic 
name of this card, for use in printouts. A typical pile might look like this: 


Actual cards Computer representation 


100: +}1/)]1 /10 A 
101: +u 10 4 C 


386: +|/0}4/3] 100 
387: t+lo u 3 u S 


242: +/0/2/2] 386 


| G 243: l+] y 2 y D 


The memory locations in the computer representation are shown here as 100, 
386, and 242; they could have been any other numbers as far as this example 
is concerned, since each card links to the one below it. Notice the special link 
“A” in node 100; we use the capital Greek letter Lambda to denote the null link, 
the link to no node. The null link A appears in node 100 since the 10 of clubs 
is the bottom card of the pile. Within the machine, A is represented by some 
easily recognizable value that cannot be the address of a node. We will generally 
assume that no node appears in location 0; consequently, A will almost always 
be represented as the link value 0 in MIX programs. 

The introduction of links to other elements of data is an extremely important 
idea in computer programming; links are the key to the representation of complex 
structures. When displaying computer representations of nodes it is usually 
convenient to represent links by arrows, so that example (2) would appear thus: 


TOP ——>}+/0/2]|2 e—>+/0/4/3 e—>+]1}1/10) e (3) 
+/u-u 2 uD +/u-u 3 u 8 +/u 1 0u cl] > 3 


The actual locations 242, 386, and 100 (which are irrelevant anyway) no longer 
appear in representation (3). Electrical circuit notation for a “grounded” wire 
is used to indicate a null link, shown here at the right of the diagram. Notice 
also that (3) indicates the top card by an arrow from “TOP”; here TOP is a link 
variable, often called a pointer variable, namely a variable whose value is a link. 
All references to nodes in a program are made directly through link variables (or 
link constants), or indirectly through link fields in other nodes. 
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Now we come to the most important part of the notation, the means of 
referring to fields within nodes. This is done simply by giving the name of the 
field followed by a link to the desired node in parentheses; for example, in (2) 
and (3) with the fields of (1) we have 


RANK(100) = 10; SUIT(TOP) = 2; 
TITLE(TOP) = “Lu2uD”; RANK (NEXT (TOP) ) = 3. 


(4) 


The reader should study these examples carefully, since such field notations 
will be used in many algorithms of this chapter and the following chapters. To 
make the ideas clearer, we will now state a simple algorithm for placing a new 
card face up on top of the pile, assuming that NEWCARD is a link variable whose 
value is a link to the new card: 


A1. Set NEXT(NEWCARD) + TOP. (This puts the appropriate link into the new 
card node.) 


A2. Set TOP + NEWCARD. (This keeps TOP pointing to the top of the pile.) 
A3. Set TAG(TOP) + 0. (This marks the card as “face up.”) q 


Another example is the following algorithm, which counts the number of cards 
currently in the pile: 


B1. Set N + 0, X + TOP. (Here N is an integer variable, X is a link variable.) 
B2. If X = A, stop; N is the number of cards in the pile. 
B3. Set N + N + 1, X + NEXT(X), and go back to step B2. J 


Notice that we are using symbolic names for two quite different things in 
these algorithms: as names of variables (TOP, NEWCARD, N, X) and as names of 
fields (TAG, NEXT). These two usages must not be confused. If F is a field name 
and L Æ A is a link, then F(L) is a variable; but F itself is not a variable— it 
does not possess a value unless it is qualified by a nonnull link. 

Two further notations are used, to convert between addresses and the values 
stored there, when we are discussing low-level machine details: 


a) CONTENTS always denotes a full-word field of a one-word node. Thus 
CONTENTS (1000) denotes the value stored in memory location 1000; it is a 
variable having this value. If V is a link variable, CONTENTS (V) denotes the 
value pointed to by V (not the value V itself). 

b) If V is the name of some value held in a memory cell, LOC(V) denotes the 
address of that cell. Consequently, if V is a variable whose value is stored in a 
full word of memory, we have CONTENTS (LOC (V)) = V. 


It is easy to transform this notation into MIXAL assembly language code, 
although MIXAL’s notation is somewhat backwards. The values of link variables 
are put into index registers, and the partial-field capability of MIX is used to refer 
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to the desired field. For example, Algorithm A above could be written thus: 


NEXT EQU 4:5 Definition of the NEXT 
TAG EQU 1:1 and TAG fields for the assembler 
LD1 NEWCARD Al. rll + NEWCARD. 
LDA TOP rA + TOP. (5) 
STA 0,1(NEXT) NEXT (rI1) + rA. 
ST1 TOP A2. TOP < rll. 


STZ 0,1(TAG) A3. TAGI) <0. I 


The ease and efficiency with which these operations can be carried out in a 
computer is the primary reason for the importance of the “linked memory” 
concept. 

Sometimes we have a single variable that denotes a whole node; its value is 
a sequence of fields instead of just one field. Thus we might write 


CARD + NODE (TOP), (6) 


where NODE is a field specification just like CONTENTS, except that it refers to an 
entire node, and where CARD is a variable that assumes structured values like 
those in (1). If there are c words in a node, the notation (6) is an abbreviation 
for the c low-level assignments 


CONTENTS (LOC (CARD) + j) 4+ CONTENTS (TOP + j), O<j<e. (7) 


There is an important distinction between assembly language and the no- 
tation used in algorithms. Since assembly language is close to the machine’s 
internal language, the symbols used in MIXAL programs stand for addresses 
instead of values. Thus in the left-hand columns of (5), the symbol TOP actually 
denotes the address where the pointer to the top card appears in memory; but 
in (6) and (7) and in the remarks at the right of (5), it denotes the value of 
TOP, namely the address of the top card node. This difference between assembly 
language and higher-level language is a frequent source of confusion for beginning 
programmers, so the reader is urged to work exercise 7. The other exercises also 
provide useful drills on the notational conventions introduced in this section. 


EXERCISES 

1. [04] In the situation depicted in (3), what is the value of (a) SUIT(NEXT(TOP)); 
(b) NEXT (NEXT (NEXT (TOP) )) ? 

2. [10] The text points out that in many cases CONTENTS (LOC(V)) = V. Under what 
conditions do we have LOC(CONTENTS(V)) = V? 


3. [11] Give an algorithm that essentially undoes the effect of Algorithm A: It 
removes the top card of the pile (if the pile is not empty) and sets NEWCARD to the 
address of this card. 


4. [18] Give an algorithm analogous to Algorithm A, except that it puts the new 
card face down at the bottom of the pile. (The pile may be empty.) 


> 5. [21] Give an algorithm that essentially undoes the effect of exercise 4: Assuming 
that the pile is not empty and that its bottom card is face down, your algorithm should 
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remove the bottom card and make NEWCARD link to it. (This algorithm is sometimes 
called “cheating” in solitaire games.) 


6. [06] In the playing card example, suppose that CARD is the name of a variable 
whose value is an entire node as in (6). The operation CARD « NODE(TOP) sets the 
fields of CARD respectively equal to those of the top of the pile. After this operation, 
which of the following notations stands for the suit of the top card? (a) SUIT(CARD); 
(b) SUIT(LOC (CARD) ); (c) SUIT(CONTENTS (CARD) ); (d) SUIT(TOP) ? 


7. [04] In the text’s example MIX program, (5), the link variable TOP is stored in the 
MIX computer word whose assembly language name is TOP. Given the field structure (1), 
which of the following sequences of code brings the quantity NEXT(TOP) into register A? 
Explain why the other sequence is incorrect. 


a) LDA TOP (NEXT) b) LD1 TOP 
LDA 0,1(NEXT) 
8. [18] Write a MIX program corresponding to steps B1-B83. 
9. [23] Write a MIX program that prints out the alphabetic names of the current 


contents of the card pile, starting at the top card, with one card per line, and with 
parentheses around cards that are face down. 
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2.2.1. Stacks, Queues, and Deques 


DATA USUALLY HAS much more structural information than we actually want to 
represent directly in a computer. For example, each “playing card” node in the 
preceding section had a NEXT field to specify what card was beneath it in the pile, 
but we provided no direct way to find what card, if any, was above a given card, 
or to find what pile a given card was in. And of course we totally suppressed 
most of the characteristic features of real playing cards: the details of the design 
on the back, the relation to other objects in the room where the game was being 
played, the individual molecules within the cards, etc. It is conceivable that such 
structural information would be relevant in certain computer applications, but 
obviously we never want to store all of the structure that is present in every 
situation. Indeed, for most card-playing situations we would not need all of the 
facts retained in the earlier example; the TAG field, which tells whether a card is 
face up or face down, will often be unnecessary. 

We must decide in each case how much structure to represent in our tables, 
and how accessible to make each piece of information. To make such decisions, 
we need to know what operations are to be performed on the data. For each 
problem considered in this chapter, therefore, we consider not only the data 
structure but also the class of operations to be done on the data; the design of 
computer representations depends on the desired function of the data as well as 
on its intrinsic properties. Indeed, an emphasis on function as well as form is 
basic to design problems in general. 

In order to illustrate this point further, let’s consider a related aspect of 
computer hardware design. A computer memory is often classified as a “random 
access memory,” like MIX’s main memory; or as a “read-only memory,” which 
is supposed to contain essentially constant information; or a “secondary bulk 
memory,” like MIX’s disk units, which cannot be accessed at high speed al- 
though large quantities of information can be stored; or an “associative memory,” 
more properly called a “content-addressed memory,” for which information is 
addressed by its value rather than by its location; and so on. The intended 
function of each kind of memory is so important that it enters into the name of 
the particular memory type; all of these devices are “memory” units, but the 
purposes to which they are put profoundly influence their design and their cost. 


A linear list is a sequence of n > 0 nodes X[1], X[2], ..., Xfm] whose 
essential structural properties involve only the relative positions between items 
as they appear in a line. The only things we care about in such structures are the 
facts that, if n > 0, X[1] is the first node and X[n] is the last; andifl<k<n, 
the kth node X[k] is preceded by X[k — 1] and followed by X[k + 1]. 

The operations we might want to perform on linear lists include, for example, 
the following. 


i) Gain access to the kth node of the list to examine and/or to change the 
contents of its fields. 
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Insert a new node just before or after the kth node. 
Delete the kth node. 
Combine two or more linear lists into a single list. 


Make a copy of a linear list. 

Determine the number of nodes in a list. 

Sort the nodes of the list into ascending order based on certain fields of the 
nodes. 


ix) Search the list for the occurrence of a node with a particular value in some 
field. 


) 
) 
) 
v) Split a linear list into two or more lists. 
) 
) 
) 


In operations (i), (ii), and (iii) the special cases k = 1 and k = n are of principal 
importance, since the first and last items of a linear list may be easier to get at 
than a general element is. We will not discuss operations (viii) and (ix) in this 
chapter, since those topics are the subjects of Chapters 5 and 6, respectively. 

A computer application rarely calls for all nine of these operations in their 
full generality, so we find that there are many ways to represent linear lists 
depending on the class of operations that are to be done most frequently. It 
is difficult to design a single representation method for linear lists in which all 
of these operations are efficient; for example, the ability to gain access to the 
kth node of a long list for random k is comparatively hard to do if at the same 
time we are inserting and deleting items in the middle of the list. Therefore we 
distinguish between types of linear lists depending on the principal operations to 
be performed, just as we have noted that computer memories are distinguished 
by their intended applications. 

Linear lists in which insertions, deletions, and accesses to values occur almost 
always at the first or the last node are very frequently encountered, and we give 
them special names: 


A stack is a linear list for which all insertions and deletions (and usually all 
accesses) are made at one end of the list. 


A queue is a linear list for which all insertions are made at one end of the 
list; all deletions (and usually all accesses) are made at the other end. 


A deque (“double-ended queue”) is a linear list for which all insertions and 
deletions (and usually all accesses) are made at the ends of the list. 


A deque is therefore more general than a stack or a queue; it has some properties 
in common with a deck of cards, and it is pronounced the same way. We also 
distinguish output-restricted or input-restricted deques, in which deletions or 
insertions, respectively, are allowed to take place at only one end. 

In some disciplines the word “queue” has been used in a much broader sense, 
to describe any kind of list that is subject to insertions and deletions; the special 
cases identified above are then called various “queuing disciplines.” Only the 
restricted use of the term “queue” is intended in this book, however, by analogy 
with orderly queues of people waiting in line for service. 
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Output from stack Input to stack 


REII SUI 


Fig. 1. A stack represented as a rail- 
way switching network. Wy 


Sometimes it helps to understand the mechanism of a stack in terms of an 
analogy from the switching of railroad cars, as suggested by E. W. Dijkstra (see 
Fig. 1). A corresponding picture for deques is shown in Fig. 2. 


This track closed on input-restricted deque 


Output E EN Deque ii l Input 
Kiman iOD anea H I Sent 
From deque To deque 


4 
This track closed on output-restricted deque 


Fig. 2. A deque represented as a railway switching network. 


With a stack we always remove the “youngest” item currently in the list, 
namely the one that has been inserted more recently than any other. With a 
queue just the opposite is true: The “oldest” item is always removed; the nodes 
leave the list in the same order as they entered it. 

Many people who have independently realized the importance of stacks and 
queues have given them other names: Stacks have been called push-down lists, 
reversion storages, cellars, nesting stores, piles, last-in-first-out (“LIFO”) lists, 
and even yo-yo lists. Queues are sometimes called circular stores or first-in-first- 
out (“FIFO”) lists. The terms LIFO and FIFO have been used for many years 
by accountants, as names of methods for pricing inventories. Still another term, 
“shelf,” has been applied to output-restricted deques, and input-restricted deques 
have been called “scrolls” or “rolls.” This multiplicity of names is interesting in 
itself, since it is evidence for the importance of the concepts. The words stack 
and queue are gradually becoming standard terminology; of all the other words 
listed above, only “push-down list” is still reasonably common, particularly in 
connection with automata theory. 

Stacks arise quite frequently in practice. We might, for example, go through 
a set of data and keep a list of exceptional conditions or things to do later; after 
we’re done with the original set, we can then do the rest of the processing by 
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coming back to the list, removing entries until it becomes empty. (The “saddle 
point” problem, exercise 1.3.2-10, is an instance of this situation.) Either a stack 
or a queue will be suitable for such a list, but a stack is generally more convenient. 
We all have “stacks” in our minds when we are solving problems: One problem 
leads to another and this leads to another; we stack up problems and subproblems 
and remove them as they are solved. Similarly, the process of entering and 
leaving subroutines during the execution of a computer program has a stack-like 
behavior. Stacks are particularly useful for the processing of languages with a 
nested structure, like programming languages, arithmetic expressions, and the 
literary German “Schachtelsatze.” In general, stacks occur most frequently in 
connection with explicitly or implicitly recursive algorithms, and we will discuss 
this connection thoroughly in Chapter 8. 
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Fig. 3. Three important classes of linear lists. 


Special terminology is generally used when algorithms refer to these struc- 
tures: We put an item onto the top of a stack, or take the top item off (see 
Fig. 3a). The bottom of the stack is the least accessible item, and it will not be 
removed until all other items have been deleted. (People often say that they push 
an item down onto a stack, and pop the stack up when the top item is deleted. 
This terminology comes from an analogy with the stacks of plates often found in 
cafeterias. The brevity of the words “push” and “pop” has its advantages, but 
these terms falsely imply a motion of the whole list within computer memory. 
Nothing is physically pushed down; items are added onto the top, as in haystacks 
or stacks of boxes.) With queues, we speak of the front and the rear of the queue; 
things enter at the rear and are removed when they ultimately reach the front po- 
sition (see Fig. 3b). When referring to deques, we speak of the left and right ends 
(Fig. 3c). The concepts of top, bottom, front, and rear are sometimes applied to 
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deques that are being used as stacks or queues, with no standard conventions as 
to whether top, bottom, front, and rear should appear at the left or the right. 
Thus we find it easy to use a rich variety of descriptive words from English in 
our algorithms: “up-down” terminology for stacks, “waiting in line” terminology 
for queues, and “left-right” terminology for deques. 
A little bit of additional notation has proved to be convenient for dealing 
with stacks and queues: We write 


A<=¢r (1) 


(when A is a stack) to mean that the value x is inserted on top of stack A, or 
(when A is a queue) to mean that x is inserted at the rear of the queue. Similarly, 
the notation 

T&A (2) 


is used to mean that the variable x is set equal to the value at the top of stack A 
or at the front of queue A, and this value is deleted from A. Notation (2) is 
meaningless when A is empty — that is, when A contains no values. 

If A is a nonempty stack, we may write 


top(A) (3) 


to denote its top element. 


EXERCISES 


1. [06] An input-restricted deque is a linear list in which items may be inserted at 
one end but removed from either end; clearly an input-restricted deque can operate 
either as a stack or as a queue, if we consistently remove all items from one of the two 
ends. Can an output-restricted deque also be operated either as a stack or as a queue? 


2. [15] Imagine four railroad cars positioned on the input side of the track in Fig. 1, 
numbered 1, 2, 3, and 4, from left to right. Suppose we perform the following sequence 
of operations (which is compatible with the direction of the arrows in the diagram and 
does not require cars to “jump over” other cars): (i) move car 1 into the stack; (ii) move 
car 2 into the stack; (iii) move car 2 into the output; (iv) move car 3 into the stack; 
(v) move car 4 into the stack; (vi) move car 4 into the output; (vii) move car 3 into 
the output; (viii) move car 1 into the output. 

As a result of these operations the original order of the cars, 1234, has been 
changed into 2431. It is the purpose of this exercise and the following exercises to 
examine what permutations are obtainable in such a manner from stacks, queues, or 
deques. 

If there are six railroad cars numbered 123456, can they be permuted into the 
order 325641? Can they be permuted into the order 154623? (In case it is possible, 
show how to do it.) 


3. [25] The operations (i) through (viii) in the previous exercise can be much more 
concisely described by the code SSXSSXXX, where S stands for “move a car from the 
input into the stack,” and X stands for “move a car from the stack into the output.” 
Some sequences of S’s and X’s specify meaningless operations, since there may be no 
cars available on the specified track; for example, the sequence SXXSSXXS cannot be 
carried out, since we assume that the stack is initially empty. 
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Let us call a sequence of S’s and X’s admissible if it contains n S’s and n X’s, and 
if it specifies no operations that cannot be performed. Formulate a rule by which it is 
easy to distinguish between admissible and inadmissible sequences; show furthermore 
that no two different admissible sequences give the same output permutation. 


4. [M34] Find a simple formula for an, the number of permutations on n elements 
that can be obtained with a stack like that in exercise 2. 


5. [M28] Show that it is possible to obtain a permutation pip2...pn from 12... 
using a stack if and only if there are no indices i < j < k such that pj < pr < pi. 

6. [00] Consider the problem of exercise 2, with a queue substituted for a stack. 
What permutations of 12... can be obtained with use of a queue? 


7. [25] Consider the problem of exercise 2, with a deque substituted for a stack. 
(a) Find a permutation of 1234 that can be obtained with an input-restricted deque, 
but it cannot be obtained with an output-restricted deque. (b) Find a permutation 
of 1234 that can be obtained with an output-restricted deque but not with an input- 
restricted deque. [As a consequence of (a) and (b), there is definitely a difference 
between input-restricted and output-restricted deques.] (c) Find a permutation of 1234 
that cannot be obtained with either an input-restricted or an output-restricted deque. 


8. [22] Are there any permutations of 12...n that cannot be obtained with the use 
of a deque that is neither input- nor output-restricted? 


9. [M20] Let bn be the number of permutations on n elements obtainable by the use 
of an input-restricted deque. (Note that b4 = 22, as shown in exercise 7.) Show that 
bn is also the number of permutations on n elements with an output-restricted deque. 


10. [M25] (See exercise 3.) Let S, Q, and X denote respectively the operations of in- 
serting an element at the left, inserting an element at the right, and emitting an element 
from the left, of an output-restricted deque. For example, the sequence QQXSXSXKX 
will transform the input sequence 1234 into 1342. The sequence SXQSXSXX gives the 
same transformation. 

Find a way to define the concept of an admissible sequence of the symbols S, Q, 
and X, so that the following property holds: Every permutation of n elements that 
is attainable with an output-restricted deque corresponds to precisely one admissible 
sequence. 


11. [M40] As a consequence of exercises 9 and 10, the number bn is the number of 
admissible sequences of length 2n. Find a closed form for the generating function 
aes pinz”. 

12. [HM34] Compute the asymptotic values of the quantities an and bn in exercises 
4 and 11. 


13. [M48] How many permutations of n elements are obtainable with the use of a 
general deque? [See Rosenstiehl and Tarjan, J. Algorithms 5 (1984), 389-390, for an 
algorithm that decides in O(n) steps whether or not a given permutation is obtainable.] 


14. [26] Suppose you are allowed to use only stacks as data structures. How can you 
implement a queue efficiently with two stacks? 
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2.2.2. Sequential Allocation 


The simplest and most natural way to keep a linear list inside a computer is to 
put the list items in consecutive locations, one node after the other. Then we 
will have 

LOC(X[j +11) = LOC(X[j]) +c, 


where c is the number of words per node. (Usually c = 1. When c > 1, it is 
sometimes more convenient to split a single list into c “parallel” lists, so that 
the kth word of node X[j] is stored a fixed distance from the location of the 
first word of X[j], depending on k. We will continually assume, however, that 
adjacent groups of c words form a single node.) In general, 


LOC(X[j1) = Lo + cj, (1) 


where Lo is a constant called the base address, the location of an artificially 
assumed node X [0]. 

This technique for representing a linear list is so obvious and well-known 
that there seems to be no need to dwell on it at any length. But we will be 
seeing many other “more sophisticated” methods of representation later on in 
this chapter, and it is a good idea to examine the simple case first to see just 
how far we can go with it. It is important to understand the limitations as well 
as the power of the use of sequential allocation. 

Sequential allocation is quite convenient for dealing with a stack. We simply 
have a variable T called the stack pointer. When the stack is empty, we let T = 0. 
To place a new element Y on top of the stack, we set 


T¢eT+1; X(T] Y. (2) 


And when the stack is not empty, we can set Y equal to the top node and delete 
that node by reversing the actions of (2): 


Y + X(T]; T-T-l1. (3) 


(Inside a computer it is usually most efficient to maintain the value cT instead 
of T, because of (1). Such modifications are easily made, so we will continue our 
discussion as though c = 1.) 

The representation of a queue or a more general deque is a little trickier. An 
obvious solution is to keep two pointers, say F and R (for the front and rear of 
the queue), with F = R = 0 when the queue is empty. Then inserting an element 
at the rear of the queue would be 


R¢R+1; X[R] < Y. (4) 
Removing the front node (F points just below the front) would be 
Fc F+1; y + X[F]; if F =R, then set F +R & 0. (5) 


But note what can happen: If R always stays ahead of F (so that there is 
always at least one node in the queue) the table entries used are X[1], X[2], 
..., X[1000], ..., ad infinitum, and this is terribly wasteful of storage space. 
The simple method of (4) and (5) should therefore be used only in the situation 
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when F is known to catch up to R quite regularly —for example, if all deletions 
come in spurts that empty the queue. 

To circumvent the problem of the queue overrunning memory, we can set 
aside M nodes X[1],...,X[M] arranged implicitly in a circle with X[1] following 
X[M]. Then processes (4) and (5) above become 


ifR=MthenR< 1, otherwise R R + 1; X[R] + Y. (6) 
ifF =M then F + 1, otherwise F F +1; Y< X[F]. (7) 


We have, in fact, already seen circular queuing action like this, when we looked 
at input-output buffering in Section 1.4.4. 

Our discussion so far has been very unrealistic, because we have tacitly 
assumed that nothing could go wrong. When we deleted a node from a stack or 
queue, we assumed that there was at least one node present. When we inserted 
a node into a stack or queue, we assumed that there was room for it in memory. 
But clearly the method of (6) and (7) allows at most M nodes in the entire queue, 
and methods (2), (3), (4), (5) allow T and R to reach only a certain maximum 
amount within any given computer program. The following specifications show 
how the actions should be rewritten for the common case where we do not assume 
that these restrictions are automatically satisfied: 


T¢T+1; 
X < Y (insert into stack): if T >M, then OVERFLOW; (2a) 
X(T] + Y. 
if T = 0, then UNDERFLOW; 
Y < X (delete from stack): ¢ Y + X(T]; (3a) 
Te Ti. 
if R = M, then R + 1, otherwise R + R + 1; 
X < Y (insert into queue): 4 if R = F, then OVERFLOW; (6a) 
X[R] = Y. 
if F = R, then UNDERFLOW; 
Y < X (delete from queue): { if F =M, then F + 1, otherwise F — F + 1; (7a) 
y + X[F]. 


Here we assume that X[1],...,X[M] is the total amount of space allowed for the 
list; OVERFLOW and UNDERFLOW mean an excess or deficiency of items. The initial 
setting F = R = 0 for the queue pointers is no longer valid when we use (6a) and 
(7a), because overflow will not be detected when F = 0; we should start with 
F =R = 1, say. 

The reader is urged to work exercise 1, which discusses a nontrivial aspect 
of this simple queuing mechanism. 

The next question is, “What do we do when UNDERFLOW or OVERFLOW oc- 
curs?” In the case of UNDERFLOW, we have tried to remove a nonexistent item; 
this is usually a meaningful condition— not an error situation— that can be 
used to govern the flow of a program. For example, we might want to delete 
items repeatedly until UNDERFLOW occurs. An OVERFLOW situation, however, is 
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usually an error; it means that the table is full already, yet there is still more 
information waiting to be put in. The usual policy in case of OVERFLOW is to 
report reluctantly that the program cannot go on because its storage capacity 
has been exceeded; then the program terminates. 

Of course we hate to give up in an OVERFLOW situation when only one list 
has gotten too large, while other lists of the same program may very well have 
plenty of room remaining. In the discussion above we were primarily thinking of 
a program with only one list. However, we frequently encounter programs that 
involve several stacks, each of which has a dynamically varying size. In such a 
situation we don’t want to impose a maximum size on each stack, since the size 
is usually unpredictable; and even if a maximum size has been determined for 
each stack, we will rarely find all stacks simultaneously filling their maximum 
capacity. 

When there are just two variable-size lists, they can coexist together very 
nicely if we let the lists grow toward each other: 


Program and Available Program and 
fixed-size tables List 1 space List 2 fixed-size tables 
Beginning Bottom Top Top Bottom End of 
of memory memory 


Here list 1 expands to the right, and list 2 (stored in reverse order) expands to 
the left. OVERFLOW will not occur unless the total size of both lists exhausts all 
memory space. The lists may independently expand and contract so that the 
effective maximum size of each one could be significantly more than half of the 
available space. This layout of memory space is used very frequently. 

We can easily convince ourselves, however, that there is no way to store three 
or more variable-size sequential lists in memory so that (a) OVERFLOW will occur 
only when the total size of all lists exceeds the total space, and (b) each list has 
a fixed location for its “bottom” element. When there are, say, ten or more 
variable-size lists—and this is not unusual—the storage allocation problem 
becomes very significant. If we wish to satisfy condition (a), we must give up 
condition (b); that is, we must allow the “bottom” elements of the lists to change 
their positions. This means that the location Lo of Eq. (1) is not constant any 
longer; no reference to the table may be made to an absolute memory address, 
since all references must be relative to the base address Lo. In the case of MIX, 
the coding to bring the Ith one-word node into register A is changed from 


LD1i I 
LD1 I a nee LDA BASE(0:2) (8) 
LDA Lo,1 2 SOE OXAMpe, STA *+1(0:2) 
LDA *,1 
where BASE contains Lo 000|. Such relative addressing evidently 


takes longer than fixed-base addressing, although it would be only slightly slower 
if MIX had an “indirect addressing” feature (see exercise 3). 
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An important special case occurs when each of the variable-size lists is a 
stack. Then, since only the top element of each stack is relevant at any time, we 
can proceed almost as efficiently as before. Suppose that we have n stacks; the 
insertion and deletion algorithms above become the following, if BASE[i] and 
TOP [i] are link variables for the ith stack, and if each node is one word long: 


Insertion: TOP[i] «+ TOP [i] +1; if TOP [i] > BASE[i+ 1], then 
OVERFLOW; otherwise set CONTENTS (TOP [i]) < Y. (9) 


Deletion: if TOP[¢] = BASE[i], then UNDERFLOW; otherwise 
set Y < CONTENTS(TOP[i]), TOP[i] + TOP[i] —1. (10) 


Here BASE[i +1] is the base location of the (i + 1)st stack. The condition 
TOP [i] = BASE[z] means that stack 2 is empty. 

In (g), OVERFLOW is no longer such a crisis as it was before; we can “repack 
memory,” making room for the table that overflowed by taking some away from 
tables that aren’t yet filled. Several ways to do the repacking suggest themselves; 
we will now consider some of them in detail, since they can be quite important 
when linear lists are allocated sequentially. We will start by giving the simplest 
of the methods, and will then consider some of the alternatives. 

Assume that there are n stacks, and that the values BASE[i] and TOP[i] 
are to be treated as in (g) and (10). These stacks are all supposed to share a 
common memory area consisting of all locations L with Lo < L < Ly. (Here Lo 
and Læ are constants that specify the total number of words available for use.) 
We might start out with all stacks empty, and 


BASE[j] = TOP [j] = Lo forl<j<n. (11) 
We also set BASE[n + 1] = Læ so that (9) will work properly for i = n. 
When OVERFLOW occurs with respect to stack i, there are three possibilities: 
a) We find the smallest k for which i < k < n and TOP[k] < BASE[k + 1], if 
any such k exist. Now move things up one notch: 


Set CONTENTS (L + 1) + CONTENTS(L), for TOP[A] > L > BASE[i+ 1]. 


(This must be done for decreasing, not increasing, values of L to avoid losing 
information. It is possible that TOP [k] = BASE[z + 1], in which case nothing 
needs to be moved.) Finally we set BASE[j] < BASE[j] +1 and TOP[j] + 
TOPLj] +1, fori<j<k. 

b) No k can be found as in (a), but we find the largest k for which 1 < k <i 
and TOP[k] < BASE[A +1]. Now move things down one notch: 


Set CONTENTS(L—1) «+ CONTENTS(L), for BASE[k +1] < L < TOP[i]. 


(This must be done for increasing values of L.) Then set BASE[j] + 
BASE[j] — 1 and TOP[j] «+ TOP[j] —1, fork <j <i. 


c) We have TOP[k] = BASE[k+ 1] for all k Æ i. Then obviously we cannot 
find room for the new stack entry, and we must give up. 
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1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 


1> 

TOP [1] —> 
BASE [2] —> 
TOP [2] —>| 
BASE [3] —> 
TOP [3] — 
BASE [4] —> 
TOP [4] —>| 
BASE [5] —> 


BASE[1 


Fig. 4. Example of memory configuration after several insertions and deletions. 


Figure 4 illustrates the configuration of memory for the case n = 4, Lo = 0, 
Loo = 20, after the successive actions 


E ELE D EL E E I D, D. (12) 


(Here I; and D; refer to insertion and deletion in stack j, and an asterisk refers 
to an occurrence of OVERFLOW, assuming that no space is initially allocated to 
stacks 1, 2, and 3.) 

It is clear that many of the first stack overflows that occur with this method 
could be eliminated if we chose our initial conditions wisely, instead of allocating 
all space initially to the nth stack as suggested in (11). For example, if we expect 
each stack to be of the same size, we can start out with 


sasti] = opti] = | (4—*) (bx to) ity, idiom G3) 


Operating experience with a particular program may suggest better starting 
values; however, no matter how well the initial allocation is set up, it can save 
at most a fixed number of overflows, and the effect is noticeable only in the early 
stages of a program run. (See exercise 17.) 

Another possible way to improve the method above would be to make room 
for more than one new entry each time memory is repacked. This idea has 
been exploited by J. Garwick, who suggests a complete repacking of memory 
when overflow occurs, based on the change in size of each stack since the last 
repacking. His algorithm uses an additional array, called OLDTOP[j], 1< j <n, 
which retains the value that TOP[j] had just after the previous allocation of 
memory. Initially, the tables are set as before, with OLDTOP [j] = TOP [j]. The 
algorithm proceeds as follows: 


Algorithm G (Reallocate sequential tables). Assume that OVERFLOW has oc- 
curred in stack i, according to (9). After Algorithm G has been performed, 
either we will find the memory capacity exceeded or the memory will have been 
rearranged so that the action CONTENTS (TOP [i] ) + Y may be done. (Notice that 
TOP [i] has already been increased in (9) before Algorithm G takes place.) 


G1. [Initialize.] Set SUM + Lə — Lo, INC + 0. Then do step G2 for 1 <j <n. 
(The effect will be to make SUM equal to the total amount of memory space 
left, and INC equal to the total amount of increases in table sizes since the 
last allocation.) After this has been done, go on to step G3. 
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G2. [Gather statistics.] Set SUM < SUM — (TOP[j] — BASE[j]). If TOP[j] > 
OLDTOP [j], set D[7] < TOP[j] — OLDTOP[j] and INC + INC + D[j]; 
otherwise set D[j] + 0. 

G3. [Is memory full?] If SUM < 0, we cannot proceed. 

G4. [Compute allocation factors.] Set a < 0.1 x SUM/n, 6 < 0.9 x SUM/INC. 
(Here a and 8 are fractions, not integers, which are to be computed to 
reasonable accuracy. The following step awards the available space to indi- 
vidual lists as follows: Approximately 10 percent of the memory presently 
available will be shared equally among the n lists, and the other 90 percent 
will be divided proportionally to the amount of increase in table size since 
the previous allocation.) 

G5. [Compute new base addresses.] Set NEWBASE[1] + BASE[1] and o + 0; 
then for j = 2, 3, ..., n set T + o +a +D[Ij-— 1], NEWBASELj] + 
NEWBASE[j — 1] + TOP[7 — 1] — BASE[j — 1] + |T] — lø], and o «Tr. 

G6. [Repack.] Set TOP [i] + TOP [i] — 1. (This reflects the true size of the ith 
list, so that no attempt will be made to move information from beyond 
the list boundary.) Perform Algorithm R below, and then reset TOP [i] + 
TOP [i] + 1. Finally set OLDTOP [j] + TOP[j] forl<j<n. J 


Perhaps the most interesting part of this whole algorithm is the general 
repacking process, which we shall now describe. Repacking is not trivial, since 
some portions of memory shift up and others shift down; it is obviously important 
not to overwrite any of the good information in memory while it is being moved. 


Algorithm R (Relocate sequential tables). For 1 < j < n, the information 
specified by BASE[j] and TOP[j] in accord with the conventions stated above 
is moved to new positions specified by NEWBASE[j], and the values of BASE[j] 
and TOP[j] are suitably adjusted. This algorithm is based on the easily verified 
fact that the data to be moved downward cannot overlap with any data that is 
to be moved upward, nor with any data that is supposed to stay put. 

R1. [Initialize.] Set j < 1. 

R2. [Find start of shift.] (Now all lists from 1 to j that were to be moved down 
have been shifted into the desired position.) Increase j in steps of 1 until 
finding either 
a) NEWBASE[j] < BASE[j]: Go to R3; or 
b) j >n: Go to R4. 

R3. [Shift list down.] Set ô + BASE[j] — NEWBASE[j]. Set CONTENTS(L — ô) + 
CONTENTS (L), for L = BASE[j] +1, BASE[j] +2, ..., TOP[j]. (It is possible 
for BASE[j] to equal TOP[j], in which case no action is required.) Set 
BASE[j] < NEWBASE[j], TOP[j] + TOPL[j] — ô. Go back to R2. 

R4. [Find start of shift.] (Now all lists from j to n that were to be moved up 
have been shifted into the desired position.) Decrease j in steps of 1 until 
finding either 
a) NEWBASE[j] > BASE[j]: Go to R5; or 
b) j = 1: The algorithm terminates. 
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R5. [Shift list up.] Set 6 + NEWBASELj] — BASE[j]. Set CONTENTS(L+ ô) + 
CONTENTS (L), for L = TOP [j], TOP [j] —1,..., BASE[j] +1. (As in step R3, 
no action may actually be needed here.) Set BASE[j] + NEWBASE[)], 
TOP[j] + TOP[j] +ô. Go back to R4. J 


Notice that stack 1 never needs to be moved. Therefore we should put the 
largest stack first, if we know which one will be largest. 


In Algorithms G and R we have purposely made it possible to have 
OLDTOP [j] = D[j] = NEWBASE[j + 1] 


for 1 < j < n; that is, these three tables can share common memory locations 
since their values are never needed at conflicting times. 


We have described these repacking algorithms for stacks, but it is clear that 
they can be adapted to any relatively addressed tables in which the current 
information is contained between BASE[j] and TOP[j]. Other pointers (for 
example, FRONT[j] and REAR[j]) could also be attached to the lists, making 
them serve as a queue or deque. See exercise 8, which considers the case of a 
queue in detail. 

The mathematical analysis of dynamic storage-allocation algorithms like 
those above is extremely difficult. Some interesting results appear in the exercises 
below, although they only begin to scratch the surface as far as the general 
behavior is concerned. 

As an example of the theory that can be derived, suppose we consider the 
case when the tables grow only by insertion; deletions and subsequent insertions 
that cancel their effect are ignored. Let us assume further that each table is 
expected to fill at the same rate. This situation can be modeled by imagining 
a sequence of m insertion operations a1, @2,...,@m, where each a; is an integer 
between 1 and n (representing an insertion on top of stack a;). For example, 
the sequence 1, 1, 2, 2, 1 means two insertions to stack 1, followed by two 
to stack 2, followed by another onto stack 1. We can regard each of the n™ 
possible specifications a1, da2,...,@m as equally likely, and then we can ask for 
the average number of times it is necessary to move a word from one location to 
another during the repacking operations as the entire table is built. For the first 
algorithm, starting with all available space given to the nth stack, this question 
is analyzed in exercise 9. We find that the average number of move operations 


required is 
1 1\ (/m 
~(1l-—— : 
eee oa 


Thus, as we might expect, the number of moves is essentially proportional to the 
square of the number of times the tables grow. The same is true if the individual 
stacks aren’t equally likely (see exercise 10). 

The moral of the story seems to be that a very large number of moves will 
be made if a reasonably large number of items is put into the tables. This 
is the price we must pay for the ability to pack a large number of sequential 
tables together tightly. No theory has been developed to analyze the average 
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behavior of Algorithm G, and it is unlikely that any simple model will be able to 
describe the characteristics of real-life tables in such an environment. However, 
exercise 18 provides a worst-case guarantee that the running time will not be 
too bad if the memory doesn’t get too full. 

Experience shows that when memory is only half loaded (that is, when the 
available space equals half the total space), we need very little rearranging of the 
tables with Algorithm G. The important thing is perhaps that the algorithm 
behaves well in the half-full case and that it at least delivers the right answers 
in the almost-full case. 

But let us think about the almost-full case more carefully. When the tables 
nearly fill memory, Algorithm R takes rather long to perform its job. And to 
make matters worse, OVERFLOW is much more frequent just before the memory 
space is used up. There are very few programs that will come close to filling 
memory without soon thereafter completely overflowing it; and those that do 
overflow memory will probably waste enormous amounts of time in Algorithms G 
and R just before memory is overrun. Unfortunately, undebugged programs will 
frequently overflow memory capacity. To avoid wasting all this time, a possible 
suggestion would be to stop Algorithm G in step G3 if SUM is less than Smin, 
where the latter is chosen by the programmer to prevent excessive repacking. 
When there are many variable-size sequential tables, we should not expect to 
make use of 100 percent of the memory space before storage is exceeded. 

Further study of Algorithm G has been made by D. S. Wise and D. C. 
Watson, BIT 16 (1976), 442-450. See also A. S. Fraenkel, Inf. Proc. Letters 8 
(1979), 9-10, who suggests working with pairs of stacks that grow towards each 
other. 


EXERCISES 


1. [15] In the queue operations given by (6a) and (7a), how many items can be in 
the queue at one time without OVERFLOW occurring? 


2. [22] Generalize the method of (6a) and (7a) so that it will apply to any deque with 
fewer than M elements. In other words, give specifications for the other two operations, 
“delete from rear” and “insert at front.” 


3. [21] Suppose that MIX is extended as follows: The I-field of each instruction is to 
have the form 81; + I2, where 0 < Iı < 8,0 < I2 < 8. In assembly language one writes 
‘OP ADDRESS,I,:I2’ or (as presently) ‘OP ADDRESS, I2’ if Iı = 0. The meaning is to 
perform first the “address modification” Iı on ADDRESS, then to perform the “address 
modification” Iz on the resulting address, and finally to perform the OP with the new 
address. The address modifications are defined as follows: 

0: M=A 
1: M = A+riIl 
2: M = A + rI2 


6: M = A +rīI6 

7: M = resulting address defined from the ‘ADDRESS , I; : I2’ fields found in location 
A. The case I; = Iz = 7 in location A is not allowed. (The reason for 
the latter restriction is discussed in exercise 5.) 
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Here A denotes the address before the operation, and M denotes the resulting address 
after the address modification. In all cases the result is undefined if the value of M 
does not fit in two bytes and a sign. The execution time is increased by one unit for 
each “indirect-addressing” (modification 7) operation performed. 

As a nontrivial example, suppose that location 1000 contains ‘NOP 1000,1:7’; 
location 1001 contains ‘NOP 1000,2’; and index registers 1 and 2 respectively contain 
1 and 2. Then the command ‘LDA 1000,7:2’ is equivalent to ‘LDA 1004’, because 


1000,7:2 = (1000,1:7) ,2 = (1001,7) ,2 = (1000,2) ,2 = 1002,2 = 1004. 


a) Using this indirect addressing feature (if necessary), show how to simplify the 
coding on the right-hand side of (8) so that two instructions are saved per reference to 
the table. How much faster is your code than (8)? 

b) Suppose there are several tables whose base addresses are stored in locations 
BASE + 1, BASE + 2, BASE+ 3, ...; how can the indirect addressing feature be used to 
bring the Ith element of the Jth table into register A in one instruction, assuming that 
I is in rll and J is in r12? 

c) What is the effect of the instruction ‘ENT4 X,7’, assuming that the (3:3)-field 
in location X is zero? 


4. [25] Assume that MIX has been extended as in exercise 3. Show how to give a 
single instruction (plus auxiliary constants) for each of the following actions: 

a) To loop indefinitely because indirect addressing never terminates. 

b) To bring into register A the value LINK(LINK(2)), where the value of link variable 
x is stored in the (0:2) field of the location whose symbolic address is X, the value 
of LINK (x) is stored in the (0:2) field of location x, etc., assuming that the (3:3) 
fields in these locations are zero. 

c) To bring into register A the value LINK(LINK(LINK(2))), under assumptions like 
those in (b). 

d) To bring into register A the contents of location rI1 + rI2 + rI3 + rI4 + rI5 + rI6. 

e) To quadruple the current value of rI6. 


> 5. [35] The extension of MIX suggested in exercise 3 has an unfortunate restriction 
that “7:7” is not allowed in an indirectly addressed location. 

a) Give an example to indicate that, without this restriction, it would probably be 
necessary for the MIX hardware to be capable of maintaining a long internal stack 
of three-bit items. (This would be prohibitively expensive hardware, even for a 
mythical computer like MIX.) 

b) Explain why such a stack is not needed under the present restriction; in other 
words, design an algorithm with which the hardware of a computer could perform 
the desired address modifications without much additional register capacity. 

c) Give a milder restriction than that of exercise 3 on the use of 7:7 that alleviates the 
difficulties of exercise 4(c), yet can be cheaply implemented in computer hardware. 


6. [10] Starting with the memory configuration shown in Fig. 4, determine which of 
the following sequences of operations causes overflow or underflow: 
(a) li; (b) Io; (c) I3; (d) ULI; (e) DeDolelele. 


7. [12] Step G4 of Algorithm G indicates a division by the quantity INC. Can INC 
ever be zero at that point in the algorithm? 


> 8. [26] Explain how to modify (g), (10), and the repacking algorithms for the case 
that one or more of the lists is a queue being handled circularly as in (6a) and (7a). 
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> 9. [M27] Using the mathematical model described near the end of the text, prove 
that Eq. (14) is the expected number of moves. (Note that the sequence 1, 1, 4, 2, 3, 
1, 2, 4, 2, 1 specifies 0+0+0+1+1+3+2+0+3+6 = 16 moves.) 


10. [M28] Modify the mathematical model of exercise 9 so that some tables are 
expected to be larger than others: Let pp be the probability that a; = k, for 1 < j < m, 
1<k<n. Thus pi+p2+-:-::+pn = 1; the previous exercise considered the special case 
pk = 1/n for all k. Determine the expected number of moves, as in Eq. (14), for this 
more general case. It is possible to rearrange the relative order of the n lists so that 
the lists expected to be longer are put to the right (or to the left) of the lists that are 
expected to be shorter; what relative order for the n lists will minimize the expected 
number of moves, based on p1, p2,..-, Pn? 


11. [M30] Generalize the argument of exercise 9 so that the first t insertions in any 
stack cause no movement, while subsequent insertions are unaffected. Thus if t = 2, 
the sequence in exercise 9 specifies 0+0+0+0+0+3+0+0+3-+46 = 12 moves. What 
is the average total number of moves under this assumption? [This is an approximation 
to the behavior of the algorithm when each stack starts with t available spaces.] 


12. [M28] The advantage of having two tables coexist in memory by growing towards 
each other, rather than by having them kept in separate independently bounded areas, 
may be quantitatively estimated (to a certain extent) as follows. Use the model of 
exercise 9 with n = 2; for each of the 2” equally probable sequences a1, @2,...,@m, let 
there be kı 1s and kz 2s. (Here kı and kə are the respective sizes of the two tables 
after the memory is full. We are able to run the algorithm with m = kı + k2 locations 
when the tables are adjacent, instead of 2 max(kı, k2) locations to get the same effect 
with separate tables.) 
What is the average value of max(k1, k2)? 


13. [HM/2] The value max(k1, k2) investigated in exercise 12 will be even greater if 
larger fluctuations in the tables are introduced by allowing random deletions as well as 
random insertions. Suppose we alter the model so that with probability p the sequence 
value a; is interpreted as a deletion instead of an insertion; the process continues until 
kı +kə (the total number of table locations in use) equals m. A deletion from an empty 
list causes no effect. 

For example, if m = 4 it can be shown that we get the following probability 
distribution when the process stops: 


(kı, k2) = (0,4) (1,3) (2,2) (3,1) (4,0) 


1 1 6—6p+2p° 1 1 
16—12p+4p? 4° «16 —12p+4p?’ 4° «16 — 12 + 4p?” 


with probability 


Thus as p increases, the difference between ki and kg tends to increase. It is not 
difficult to show that in the limit as p approaches unity, the distribution of kı becomes 
essentially uniform, and the limiting expected value of max(kı, k2) is exactly am + 
z;[m odd]. This behavior is quite different from that in the previous exercise (when 
p = 0); however, it may not be extremely significant, since when p approaches unity, 
the amount of time taken to terminate the process rapidly approaches infinity. The 
problem posed in this exercise is to examine the dependence of max(k1, k2) on p and 
m, and to determine asymptotic formulas for fixed p (like p = 3) as m approaches 


infinity. The case p = ł is particularly interesting. 
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14. [HM43] Generalize the result of exercise 12 to arbitrary n > 2, by showing that, 
when n is fixed and m approaches infinity, the quantity 


m! >D max(kı, k2,..., kn) 
kı! kel... kn! 


m 
kytkote-+k,=m 
ky kay. kn 20 


has the asymptotic form m/n+cn.W/m+O(1). Determine the constants c2, c3, ca, and cs. 


15. [40] Using a Monte Carlo method, simulate the behavior of Algorithm G under 
varying distributions of insertions and deletions. What do your experiments imply 
about the efficiency of Algorithm G? Compare its performance with the algorithm 
given earlier that shifts up and down one node at a time. 


16. [20] The text illustrates how two stacks can be located so they grow towards each 
other, thereby making efficient use of a common memory area. Can two queues, or a 
stack and a queue, make use of a common memory area with the same efficiency? 


17. [30] Ifo is any sequence of insertions and deletions such as (12), let so(a) be the 
number of stack overflows that occur when the simple method of Fig. 4 is applied to ø 
with initial conditions (11), and let s:(a) be the corresponding number of overflows 
with respect to other initial conditions such as (13). Prove that so(o) < s1(a7)+Loo—Lo. 


18. [M30] Show that the total running time for any sequence of m insertions and/or 
deletions by Algorithms G and R is O(m+n X}; ax/(1—ar)), where ax is the fraction 
of memory occupied on the most recent repacking previous to the kth operation; a, = 0 
before the first repacking. (Therefore if the memory never gets more than, say, 90% 
full, each operation takes at most O(n) units of time in an amortized sense, regardless 
of the total memory size.) Assume that Loo — Lo > n?. 


19. [16] (0-origin indexing.) Experienced programmers learn that it is generally wise 
to denote the elements of a linear list by X[0], X[1], ..., Xfm — 1], instead of using the 
more traditional notation X[1], X[2], ..., X[n]. Then, for example, the base address 
Lo in (1) points to the smallest cell of the array. 

Revise the insertion and deletion methods (2a), (3a), (6a), and (7a) for stacks and 
queues so that they conform to this convention. In other words, change them so that 
the list elements will appear in the array X[0], X[1], ..., X[M — 1], instead of X[1], 
X[2],..., XM). 


2.2.3. Linked Allocation 


Instead of keeping a linear list in sequential memory locations, we can make use 
of a much more flexible scheme in which each node contains a link to the next 
node of the list. 


Sequential allocation: Linked allocation: 
Address Contents Address Contents 
Lo +e: Item 1 A: Item 1 B 
Lo + 2c: Item 2 B Item 2 C 
Lo + 3c: Item 3 C Item 3 D 
Lo + 4c: Item 4 D: Item 4 E 
Lo + 5c: Item 5 E Item 5 A 
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Here A, B, C, D, and E are arbitrary locations in the memory, and A is the null 
link (see Section 2.1). The program that uses this table in the case of sequential 
allocation would have an additional variable or constant whose value indicates 
that the table is five items in length, or else this information would be specified 
by a sentinel code within item 5 or in the following location. A program for 
linked allocation would have a link variable or link constant that points to A; all 
the other items of the list can be found from address A. 

Recall from Section 2.1 that links are often shown simply by arrows, since 
the actual memory locations occupied are usually irrelevant. The linked table 
above might therefore be shown as follows: 


FIRST ——> Item 1 | e+—> Item 2 | e+ > Item 3 | ——> Item 4 | e+ Item 5 I: 


(1) 
Here FIRST is a link variable pointing to the first node of the list. 

We can make several obvious comparisons between these two basic forms of 
storage: 

1) Linked allocation takes up additional memory space for the links. This 
can be the dominating factor in some situations. However, we frequently find 
that the information in a node does not take up a whole word anyway, so there 
is already space for a link field present. Also, it is possible in many applications 
to combine several items into one node so that there is only one link for several 
items of information (see exercise 2.5-2). But even more importantly, there is 
often an implicit gain in storage by the linked memory approach, since tables 
can overlap, sharing common parts; and in many cases, sequential allocation will 
not be as efficient as linked allocation unless a rather large number of additional 
memory locations are left vacant anyway. For example, the discussion at the end 
of the previous section explains why the systems described there are necessarily 
inefficient when memory is densely loaded. 

2) It is easy to delete an item from within a linked list. For example, to 
delete item 3 we need only change the link associated with item 2. But with 
sequential allocation such a deletion generally implies moving a large part of the 
list up into different locations. 

3) It is easy to insert an item into the midst of a list when the linked scheme 
is being used. For example, to insert an item 25 into (1) we need to change only 
two links: 


FIRST — Item 1 |A- Item 2 | ¢ Item 3 | © Item 4 Item 5 | @ 


Item 24 é 


(2) 
By comparison, this operation would be extremely time-consuming in a long 
sequential table. 

4) References to random parts of the list are much faster in the sequential 
case. To gain access to the kth item in the list, when k is a variable, takes a fixed 
time in the sequential case, but we need k iterations to march down to the right 
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place in the linked case. Thus the usefulness of linked memory is predicated on 
the fact that in the large majority of applications we want to walk through lists 
sequentially, not randomly; if items in the middle or at the bottom of the list 
are needed, we try to keep an additional link variable or list of link variables 
pointing to the proper places. 

5) The linked scheme makes it easier to join two lists together, or to break 
one apart into two that will grow independently. 

6) The linked scheme lends itself immediately to more intricate structures 
than simple linear lists. We can have a variable number of variable-size lists; 
any node of the list may be a starting point for another list; the nodes may 
simultaneously be linked together in several orders corresponding to different 
lists; and so on. 

7) Simple operations, like proceeding sequentially through a list, are slightly 
faster for sequential lists on many computers. For MIX, the comparison is between 
‘INC1 c’ and ‘LD1 0,1(LINK)’, which is only one cycle different, but many 
machines do not enjoy the property of being able to load an index register from 
an indexed location. If the elements of a linked list belong to different pages in 
a bulk memory, the memory accesses might take significantly longer. 

Thus we see that the linking technique, which frees us from any constraints 
imposed by the consecutive nature of computer memory, gives us a good deal 
more efficiency in some operations, while we lose some capabilities in other cases. 
It is usually clear which allocation technique will be most appropriate in a given 
situation, and both methods are often used in different lists of the same program. 

In the next few examples we will assume for convenience that a node has 
one word and that it is broken into the two fields INFO and LINK: 


INFO 
1 


LINK | . (3) 


The use of linked allocation generally implies the existence of some mecha- 
nism for finding empty space available for a new node, when we wish to insert 
some newly created information onto a list. This is usually done by having a 
special list called the list of available space. We will call it the AVAIL list (or the 
AVAIL stack, since it is usually treated in a last-in-first-out manner). The set of 
all nodes not currently in use is linked together in a list just like any other list; 
the link variable AVAIL refers to the top element of this list. Thus, if we want 
to set link variable X to the address of a new node, and to reserve that node for 
future use, we can proceed as follows: 


X< AVAIL, AVAIL 4+ LINK(AVAIL). (4) 


This effectively removes the top of the AVAIL stack and makes X point to the 
node just removed. Operation (4) occurs so often that we have a special notation 
for it: “X < AVAIL” will mean X is set to point to a new node. 

When a node is deleted and no longer needed, process (4) can be reversed: 


LINK(X) 4 AVAIL, AVAIL + X. (5) 
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This operation puts the node addressed by X back onto the list of raw material; 
we denote (5) by “AVAIL = X”. 

Several important things have been omitted from this discussion of the AVAIL 
stack. We did not say how to set it up at the beginning of a program; clearly 
this can be done by (a) linking together all nodes that are to be used for linked 
memory, (b) setting AVAIL to the address of the first of these nodes, and (c) 
making the last node link to A. The set of all nodes that can be allocated is 
called the storage pool. 

A more important omission in our discussion was the test for overflow: We 
neglected to check in (4) if all available memory space has been taken. The 
operation X < AVAIL should really be defined as follows: 


if AVAIL = A, then OVERFLOW; 
otherwise X < AVAIL, AVAIL «+ LINK (AVAIL). (6) 


The possibility of overflow must always be considered. Here OVERFLOW generally 
means that we terminate the program with regrets; or else we can go into a 
“garbage collection” routine that attempts to find more available space. Garbage 
collection is discussed in Section 2.3.5. 

There is another important technique for handling the AVAIL stack: We 
often do not know in advance how much memory space should be used for the 
storage pool. There may be a sequential table of variable size that wants to 
coexist in memory with the linked tables; in such a case we do not want the 
linked memory area to take any more space than is absolutely necessary. So 
suppose that we wish to place the linked memory area in ascending locations 
beginning with Lo and that this area is never to extend past the value of variable 
SEQMIN (which represents the current lower bound of the sequential table). Then 
we can proceed as follows, using a new variable POOLMAX: 


a) Initially set AVAIL ~ A and POOLMAX < Lo. 
b) The operation X < AVAIL becomes the following: 


“If AVAIL Æ A, then X + AVAIL, AVAIL + LINK(AVAIL). 
Otherwise set X + POOLMAX and POOLMAX + X + c, where c is the (7) 
node size; OVERFLOW now occurs if POOLMAX > SEQMIN.” 


c) When other parts of the program attempt to decrease the value of SEQMIN, 
they should sound the OVERFLOW alarm if SEQMIN < POOLMAX. 
d) The operation AVAIL < X is unchanged from (5). 


This idea actually represents little more than the previous method with a special 
recovery procedure substituted for the OVERFLOW situation in (6). The net 
effect is to keep the storage pool as small as possible. Many people like to 
use this idea even when all lists occupy the storage pool area (so that SEQMIN is 
constant), since it avoids the rather time-consuming operation of initially linking 
all available cells together and it facilitates debugging. We could, of course, put 
the sequential list on the bottom and the pool on the top, having POOLMIN and 
SEQMAX instead of POOLMAX and SEQMIN. 
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Thus it is quite easy to maintain a pool of available 
nodes, in such a way that free nodes can efficiently be 
found and later returned. These methods give us a source 
of raw material to use in linked tables. Our discussion 
was predicated on the implicit assumption that all nodes ? 
have a fixed size, c; the cases that arise when different x 


sizes of nodes are present are very important, but we 
will defer that discussion until Section 2.5. Now we will 
consider a few of the most common list operations in the 
special case where stacks and queues are involved. 

The simplest kind of linked list is a stack. Figure 5 
shows a typical stack, with a pointer T to the top of the T + 
stack. When the stack is empty, this pointer will have 
the value A. 

It is clear how to insert (“push down”) new information Y onto the top of 
such a stack, using an auxiliary pointer variable P. 


Fig. 5. A linked stack. 


P< AVAIL, INFO(P) +Y,  LINK(P)¢T, TP. (8) 


Conversely, to set Y equal to the information at the top of the stack and to “pop 
up” the stack: 


If T = A, then UNDERFLOW; 
otherwise set P + T, T 4 LINK(P), Y 4 INFO(P), AVAIL =P. (9) 


These operations should be compared with the analogous mechanisms for se- 
quentially allocated stacks, (2a) and (3a) in Section 2.2.2. The reader should 
study (8) and (9) carefully, since they are extremely important operations. 

Before looking at the case of queues, let us see how the stack operations can 
be expressed conveniently in programs for MIX. A program for insertion, with 
P =rI1, can be written as follows: 


INFO EQU 0:3 Definition of the INFO field 
LINK EQU 4:5 Definition of the LINK field 
LD1 AVAIL P + AVAIL. 


J1Z OVERFLOW Is AVAIL = A? 


LDA 0,1(LINK) eae 

STA AVAIL AVAIL + LINK(P). (10) 
LDA Y 

STA 0O,1(INFO) INFO(P) + Y. 

LDA T 

STA 0,1(LINK) LINK(P) ¢T. 

STi T T<P. I 


This takes 17 units of time, compared to 12 units for the comparable operation 
with a sequential table (although OVERFLOW in the sequential case would in many 
cases take considerably longer). In this program, as in others to follow in this 
chapter, OVERFLOW denotes either an ending routine or a subroutine that finds 
more space and returns to location rJ — 2. 
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A program for deletion is equally simple: 


LD1 T PeT. 
J1Z UNDERFLOW IsT= A? 
LDA 0,1(LINK) 


STA T T + LINK (P). 

LDA 0,1(INFO) (11) 
STA Y Y + INF0(P). 

LDA AVAIL 

STA 0,1(LINK) LINK(P) + AVAIL. > AVAIL < P 

ST1 AVAIL AVAIL < P. I 


It is interesting to observe that each of these operations involves a cyclic 
permutation of three links. For example, in the insertion operation let P be the 
value of AVAIL before the insertion; if P 4 A, we find that after the operation 


the value of AVAIL has become the previous value of LINK(P), 
the value of LINK (P) has become the previous value of T, and 
the value of T has become the previous value of AVAIL. 


So the insertion process (except for setting INFO(P) < Y) is the cyclic permuta- 
vioh AVAIL 


CON 


LINK (P) 


Similarly in the case of deletion, where P has the value of T before the operation 
and we assume that P #4 A, we have Y + INFO(P) and 


AVAIL 


CN 


LINK (P) 


The fact that the permutation is cyclic is not really a relevant issue, since any 
permutation of three elements that moves every element is cyclic. The important 
point is rather that precisely three links are permuted in these operations. 

The insertion and deletion algorithms of (8) and (9) have been described 
for stacks, but they apply much more generally to insertion and deletion in any 
linear list. Insertion, for example, is pee aed just before the node pointed to 
by link variable T. The insertion of item 24 in (2) above would be done by using 
operation (8) with T = LINK (LINK (FIRST)). 

Linked allocation applies in a particularly convenient way to queues. In this 
case it is easy to see that the links should run from the front of the queue towards 
the rear, so that when a node is removed from the front, the new front node is 
directly specified. We will make use of pointers F and R, to the front and rear: 


r —> > +H 7 fin (12) 


Except for R, this diagram is abstractly identical to Fig. 5 on page 258. 
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Whenever the layout of a list is designed, it is important to specify all 
conditions carefully, particularly for the case when the list is empty. One of 
the most common programming errors connected with linked allocation is the 
failure to handle empty lists properly; the other common error is to forget 
about changing some of the links when a structure is being manipulated. In 
order to avoid the first type of error, we should always examine the “boundary 
conditions” carefully. To avoid making the second type of error, it is helpful to 
draw “before and after” diagrams and to compare them, in order to see which 
links must change. 

Let’s illustrate the remarks of the preceding paragraph by applying them to 
the case of queues. First consider the insertion operation: If (12) is the situation 
before insertion, the picture after insertion at the rear of the queue should be 


r — J J J ' R 
LL (13) 


AVAIL > Y ° 


(The notation used here implies that a new node has been obtained from the 
AVAIL list.) Comparing (12) and (13) shows us how to proceed when inserting 
the information Y at the rear of the queue: 


P< AVAIL, INFO(P) 4Y, LINK(P)+ A, LINK(R) <P, R&P. (14) 


Let us now consider the “boundary” situation when the queue is empty: In 
this case the situation before insertion is yet to be determined, and the situation 
“after” is 


mr L (15) 
AVAIL 

It is desirable to have operations (14) apply in this case also, even if insertion 
into an empty queue means that we must change both F and R, not only R. 
We find that (14) will work properly if R = LOC(F) when the queue is empty, 
assuming that F = LINK(LOC(F)); the value of variable F must be stored in the 
LINK field of its location if this idea is to work. In order to make the testing for 
an empty queue as efficient as possible, we will let F = A in this case. Our policy 
is therefore that 


an empty queue is represented by F = A and R = LOC(F). 


If the operations (14) are applied under these circumstances, we obtain (15). 
The deletion operation for queues is derived in a similar fashion. If (12) is 
the situation before deletion, the situation afterwards is 


EEEH 


v 
AVAIL 


pe (16) 


For the boundary conditions we must make sure that the deletion operation 
works when the queue is empty either before or after the operation. These 
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considerations lead us to the following way to do queue deletion in general: 


If F = A, then UNDERFLOW; 
otherwise set P< F, F + LINK(P), Y + INFO(P), AVAIL =P, (17) 
and if F = A, then set R + LOC(F). 


Notice that R must be changed when the queue becomes empty; this is precisely 
the type of “boundary condition” we should always be watching for. 

These suggestions are not the only way to represent queues in a linearly 
linked fashion; exercise 30 describes a somewhat more natural alternative, and 
we will give other methods later in this chapter. Indeed, none of the operations 
above are meant to be prescribed as the only way to do something; they are 
intended as examples of the basic means of operating with linked lists. The 
reader who has had only a little previous experience with such techniques will 
find it helpful to reread the present section up to this point before going on. 


So far in this chapter we have discussed how to perform certain operations 
on tables, but our discussions have always been “abstract,” in the sense that 
we never exhibited actual programs in which the particular techniques were 
useful. People aren’t generally motivated to study abstractions of a problem until 
they’ve seen enough special instances of the problem to arouse their interest. The 
operations discussed so far — manipulations of variable-size lists of information 
by insertion and deletion, and the use of tables as stacks or queues — are of such 
wide application, it is hoped that the reader will have encountered them often 
enough already to grant their importance. But now we will leave the realm of 
the abstract as we begin to study a series of significant practical examples of the 
techniques of this chapter. 

Our first example is a problem called topological sorting, which is an impor- 
tant process needed in connection with network problems, with so-called PERT 
charts, and even with linguistics; in fact, it is of potential use whenever we have 
a problem involving a partial ordering. A partial ordering of a set S is a relation 
between the objects of S, which we may denote by the symbol “<”, satisfying 
the following properties for any objects x, y, and z (not necessarily distinct) 
in S: 

i) If x < y and y X z, then x < z. (Transitivity.) 

ii) If x < y and y < x, then x = y. (Antisymmetry.) 

iii) z < x. (Reflexivity.) 
The notation x < y may be read “x precedes or equals y.” If x < y and z Æ y, 
we write z < y and say “x precedes y.” It is easy to see from (i), (ii), and (iii) 
that we always have 

i’) If x< y and y <x z, then z < z. (Transitivity.) 

ii’) If a < y, then y £ x. (Asymmetry.) 
iii’) x Ax. (Irreflexivity.) 
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The relation denoted by y A x means “y does not precede x.” If we start with 
a relation < satisfying properties (i’), (ii’), and (iii’), we can reverse the process 
above and define x < y if x < y or x = y; then properties (i), (ii), and (iii) are 
true. Therefore we may regard either properties (i), (ii), (iii) or properties (i’), 
(ii’), (iii’) as the definition of partial order. Notice that property (ii’) is actually 
a consequence of (i’) and (iii’), although (ii) does not follow from (i) and (iii). 

Partial orderings occur quite frequently in everyday life as well as in math- 
ematics. As examples from mathematics we can mention the relation x < y 
between real numbers x and y; the relation x C y between sets of objects; 
the relation x«\y (x divides y) between positive integers. In the case of PERT 
networks, S is a set of jobs that must be done, and the relation “x ~ y” means 
“x must be done before y.” 


Vdd 


Fig. 6. A partial ordering. 


We will naturally assume that S is a finite set, since we want to work with S 
inside a computer. A partial ordering on a finite set can always be illustrated 
by drawing a diagram such as Fig. 6, in which the objects are represented by 
small boxes and the relation is represented by arrows between these boxes; £ < y 
means there is a path from the box labeled x to box y that follows the direction 
of the arrows. Property (ii) of partial ordering means there are no closed loops 
(no paths that close on themselves) in the diagram. If an arrow were drawn from 
4 to 1 in Fig. 6, we would no longer have a partial ordering. 

The problem of topological sorting is to embed the partial order in a linear 
order; that is, to arrange the objects into a linear sequence a1a2 ...an such that 
whenever a; < az, we have j < k. Graphically, this means that the boxes are to 
be rearranged into a line so that all arrows go towards the right (see Fig. 7). It 
is not immediately obvious that such a rearrangement is possible in every case, 
although such a rearrangement certainly could not be done if any loops were 
present. Therefore the algorithm we will give is interesting not only because it 
does a useful operation, but also because it proves that this operation is possible 
for every partial ordering. 

As an example of topological sorting, imagine a large glossary containing 
definitions of technical terms. We can write w2 < w if the definition of word w1 
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1 J3 J7 Ja 9—2 5 s t 6 


Fig. 7. The ordering relation of Fig. 6 after topological sorting. 


depends directly or indirectly on that of word wə. This relation is a partial 
ordering provided that there are no “circular” definitions. The problem of 
topological sorting in this case is to find a way to arrange the words in the 
glossary so that no term is used before it has been defined. Analogous problems 
arise in writing programs to process the declarations in certain assembly and 
compiler languages; they also arise in writing a user’s manual describing a 
computer language or in writing textbooks about information structures. 

There is a very simple way to do topological sorting: We start by taking an 
object that is not preceded by any other object in the ordering. This object may 
be placed first in the output. Now we remove this object from the set S; the 
resulting set is again partially ordered, and the process can be repeated until the 
whole set has been sorted. For example, in Fig. 6 we could start by removing 1 
or 9; after 1 has been removed, 3 can be taken, and so on. The only way in which 
this algorithm could fail would be if there were a nonempty partially ordered set 
in which every element was preceded by another; for in such a case the algorithm 
would find nothing to do. But if every element is preceded by another, we could 
construct an arbitrarily long sequence b1, b2,b3,... in which bj+1 < bj. Since S 
is finite, we must have b; = by for some j < k; but j < k implies that by < bj+1, 
hence b; = b, contradicts (ii). 

In order to implement this process efficiently by computer, we need to 
be ready to perform the actions described above, namely to locate objects 
that are not preceded by any others, and to remove them from the set. Our 
implementation is also influenced by the desired input and output characteristics. 
The most general program would accept alphabetic names for the objects and 
would allow gigantic sets of objects to be sorted — more than could possibly fit 
in the computer memory at once. Such complications would obscure the main 
points we are trying to make here, however; the handling of alphabetic data can 
be done efficiently by using the methods of Chapter 6, and the handling of large 
networks is left as an interesting project for the reader. 

Therefore we will assume that the objects to be sorted are numbered from 1 
to n in any order. The input of the program will be on tape unit 1: Each tape 
record contains 50 pairs of numbers, where the pair (j,k) means that object j 
precedes object k. The first pair, however, is (0,n), where n is the number of 
objects. The pair (0,0) terminates the input. We shall assume that n plus the 
number of relation pairs will fit comfortably in memory; and we shall assume 
that it is not necessary to check the input for validity. The output is to be the 
numbers of the objects in sorted order, followed by the number 0, on tape unit 2. 
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As an example of the input, we might have the relations 
9<2, 3x7, 7x5, 5<8, 8x6, 4<6, 1x3, 7x4, 9x5, 2x8. (18) 


It is not necessary to give any more relations than are needed to characterize 
the desired partial ordering. Thus, additional relations like 9 < 8 (which can 
be deduced from 9 < 5 and 5 < 8) may be omitted from or added to the input 
without harm. In general, it is necessary to give only the relations corresponding 
to arrows on a diagram such as Fig. 6. 

The algorithm that follows uses a sequential table X[1], X[2], ..., X[n], 
and each node X[k] has the form 


+ | o |COUNT[k]] TOP [k] 


Here COUNT[K] is the number of direct predecessors of object k (the number of 
relations j < k that have appeared in the input), and TOP[é] is a link to the 
beginning of the list of direct successors of object k. The latter list contains 
entries in the format 


+ | 36 suc NEXT |, 


where SUC is a direct successor of k and NEXT is the next item of the list. As an 


example of these conventions, Fig. 8 shows the schematic contents of memory 
corresponding to the input (18). 
k 1 2 3 4 5 6 7 8 9 
COUNT [K] 0 1 1 1 2 2 1 2 0 
TOP [k] ry ° ° b ° ° ° 
Vv y y y y = y y | 
SUC 3 8 7 6 8 4 6 5 
NEXT $ ° ° $ (3 ° ° 
SUC 5 2 
NEXT * " 


Fig. 8. Computer representation of Fig. 6 corresponding to the relations (18). 


Using this memory layout, it is not difficult to work out the algorithm. 
We want to output the nodes whose COUNT field is zero, then to decrease the 
COUNT fields of all successors of those nodes by one. The trick is to avoid doing 
any “searching” for nodes whose COUNT field is zero, and this can be done by 
maintaining a queue containing those nodes. The links for this queue are kept 
in the COUNT field, which by now has served its previous purpose; for clarity in 
the algorithm below, we use the notation QLINK[k] to stand for COUNT [k] when 
that field is no longer being used to keep a count. 
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Algorithm T (Topological sort). This algorithm inputs a sequence of relations 
j < k, indicating that object 7 precedes object k in a certain partial ordering, 
assuming that 1 < j,k < n. The output is the set of n objects embed- 
ded in a linear order. The internal tables used are: QLINK[0], COUNT[1] = 
QLINK [1], COUNT[2] = QLINK[2], ..., COUNT[n] = QLINK[n]; TOP[1], TOP[2], 
..., TOP[n]; a storage pool with one node for each input relation and with SUC 
and NEXT fields as shown above; P, a link variable used to refer to the nodes in 
the storage pool; F and R, integer-valued variables used to refer to the front and 
rear of a queue whose links are in the QLINK table; and N, a variable that counts 
how many objects have yet to be output. 


T1. [Initialize.] Input the value of n. Set COUNT[k] + 0 and TOP[k] «+ A for 
1l<k<n. SetNen. 


T2. [Next relation.] Get the next relation “j < k” from the input; if the input 
has been exhausted, however, go to T4. 


T3. [Record the relation.] Increase COUNT [k] by one. Set 
P < AVAIL, SUC(P) + k, NEXT(P) + TOP[j], TOP[j] +P. 


(This is operation (8).) Go back to T2. 

T4. [Scan for zeros.] (At this point we have completed the input phase; the input 
(18) would now have been transformed into the computer representation 
shown in Fig. 8. The next job is to initialize the queue of output, which 
is linked together in the QLINK field.) Set R + 0 and QLINK[0] + 0. For 
1 < k < n examine COUNT[K], and if it is zero, set QLINK[R] + k and 
R < k. After this has been done for all k, set F + QLINK[O] (which will 
contain the first value k encountered for which COUNT [k] was zero). 


T5. [Output front of queue.] Output the value of F. If F = 0, go to T8; otherwise, 
set N + N — 1, and set P ¢ TOP[F]. (Since the QLINK and COUNT tables 
overlap, we have QLINK[R] = 0; therefore the condition F = 0 occurs when 
the queue is empty.) 

T6. [Erase relations.] If P = A, go to T7. Otherwise decrease COUNT [SUC(P)] 
by one, and if it has thereby gone down to zero, set QLINK[R] < SUC(P) 
and R + SUC(P). Set P + NEXT(P) and repeat this step. (We are removing 
all relations of the form “F < k” for some k from the system, and putting 
new nodes into the queue when all their predecessors have been output.) 


T7. [Remove from queue.] Set F + QLINK[F] and go back to T5. 


T8. [End of process.] The algorithm terminates. If N = 0, we have output all of 
the object numbers in the desired “topological order,” followed by a zero. 
Otherwise the N object numbers not yet output contain a loop, in violation 
of the hypothesis of partial order. (See exercise 23 for an algorithm that 
prints out the contents of one such loop.) 1 


The reader will find it helpful to try this algorithm by hand on the input 
(18). Algorithm T shows a nice interplay between sequential memory and linked 
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memory techniques. Sequential memory is used for the main table X[1], ..., 
X[n], which contains the COUNT[k] and TOP[k] entries, because we want to 
make references to “random” parts of this table in step T3. (If the input were 
alphabetic, however, another type of table would be used for speedier search, as 
in Chapter 6.) Linked memory is used for the tables of “immediate successors,” 
since those table entries have no particular order in the input. The queue of 
nodes waiting to be output is kept in the midst of the sequential table by linking 
the nodes together in output order. This linking is done by table index instead 
of by address; in other words, when the front of the queue is X[k], we have 
F = k instead of F = LOC(X[k]). The queue operations used in steps T4, T6, 
and T7 are not identical to those in (14) and (17), since we are taking advantage 
of special properties of the queue in this system; no nodes need to be created or 
returned to available space during this part of the algorithm. 

The coding of Algorithm T in MIX assembly language has a few additional 
points of interest. Since no deletion from tables is made in the algorithm (because 
no storage must be freed for later use), the operation P < AVAIL can be done 
in an extremely simple way, as shown in lines 19 and 32 below; we need not 
keep any linked pool of memory, and we can choose new nodes consecutively. 
The program includes complete input and output with magnetic tape, according 
to the conventions mentioned above, but buffering is omitted for the sake of 
simplicity. The reader should not find it very difficult to follow the details of 
the coding in this program, since it corresponds directly to Algorithm T but 
with slight changes for efficiency. The efficient use of index registers, which is an 
important aspect of linked memory processing, is illustrated here. 


Program T (Topological sort). In this program, the following equivalences 
should be noted: rI6 = N, rl5 = buffer pointer, rl4 = k, rl8 = j and R, rl2 = 
AVAIL and P, rll = F, TOP [j] = X+ (4:5), COUNT[A] = QLINK[k] = X+k(2:3). 


01 * BUFFER AREA AND FIELD DEFINITIONS 


02 COUNT EQU 2:3 Definition of symbolic 

03 QLINK EQU 2:3 names of fields 

04 TOP EQU 4:5 

05 SUC EQU 2:3 

06 NEXT EQU 4:5 

O7 TAPEIN EQU 1 Input is on tape unit 1 

08 TAPEOUT EQU 2 Output is on tape unit 2 

09 BUFFER ORIG *+100 Tape buffer area 

10 CON -1 Sentinel at end of buffer 

11 »* INPUT PHASE 

12 TOPSORT IN  BUFFER(TAPEIN) 1 T1. Initialize. Read in the first 

13 JBUS *(TAPEIN) tape block; wait for completion. 
14 1H LD6 BUFFER+1 1 Nen. 

15 ENT4 0,6 1 

16 STZ X,4 n+1 Set COUNT[k] + 0 and TOP[k] <« A, 
17 DEC4 1 n+1 forO<k<n. 

18 JANN +*-2 n+1 (Anticipate QLINK[0] + 0 in step T4.) 
19 ENT2 X,6 1 Available storage starts after X[n]. 
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2H 


3H 


4H 


4H 


T1. Initialize 
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Vv 
T2. Next relation > T3. Record 
K— the relation 
No more 
Vv 
T4. Scan T5. Output front T6. Erase 
for zeros > of queue > relations 
Empty x more 
Vv 
T8. End of T7. Remove 
process from queue 
Vv 


Fig. 9. Topological sorting. 


ENTS BUFFER+2 
LD3 0,5 

J3P 3F 

J3Z 4F 

IN  BUFFER(TAPEIN) 
JBUS *(TAPEIN) 
ENTS BUFFER 
JMP 2B 

LD4 1,5 

LDA X,4(COUNT) 
INCA 1 

STA X,4(COUNT) 
INC2 1 

LDA X,3(TOP) 
STA 0,2(NEXT) 
ST4 0,2(SUC) 
ST2 X,3(TOP) 
INC5 2 

JMP 2B 

IOC O(TAPEIN) 
ENT4 0,6 

ENT5 -100 

ENT3 0 

LDA X,4(COUNT) 
JAP *+3 

ST4 X,3(QLINK) 
ENT3 0,4 

DEC4 1 

J4P 4B 


b-1 


œ 
| 
= 


sssareszssrernrrełjzł3š 33333383 


Prepare to read the first pair (j, k). 
T2. Next relation. 
Is j > 0? 
Is input exhausted? 
Sentinel sensed; read another 

tape block, wait for completion. 
Reset the buffer pointer. 


T3. Record the relation. 
COUNT [k] 

+1 

— COUNT[k]. 

AVAIL ¢+ AVAIL + 1. 
TOP [j] 

— NEXT(P). 
k — SUC(P). 
P — TOP[Ĵ]. 
Increase buffer pointer. 


Rewind the input tape. 

T4. Scan for zeros. k4- n. 
Reset buffer pointer for output. 
R0. 

Examine COUNT[K]. 

Is it nonzero? 

QLINK[R] + k. 

Rek. 
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49 * SORTING PHASE 


50 LD1 X(QLINK) 1 F + QLINK[O]. 

51 5H JBUS *(TAPEOUT) T5. Output front of queue. 
52 ST1 BUFFER+100,5 n+1 Store F in buffer area. 

53 JiZ 8F n+1 Is F zero? 

54 INC5 1 n Advance buffer pointer. 

55 J5N *+3 n Test if buffer is full. 

56 OUT BUFFER(TAPEOUT) c—1 If so, output a tape block. 
57 ENT5 -100 c—1 Reset the buffer pointer. 
58 DEC6 1 n N¢N-1. 

59 LD2 X,1(TOP) n P + TOP[F]. 

60 J2Z TF n T6. Erase relations. 

61 6H LD4 0,2(SUC) m rl4 + SUC(P). 

62 LDA X,4(COUNT) m COUNT [rI4] 

68 DECA 1 m —1 

64 STA X,4(COUNT) m — COUNT [rI4]. 

65 JAP *+3 m Has zero been reached? 

66 ST4 X,3(QLINK) n—a_ Ifso, set QLINK[R] + rl4. 
67 ENT3 0,4 n-a R+ r4. 

68 LD2 0,2(NEXT) m P «+ NEXT(P). 

69 J2P 6B m If P Æ A, repeat. 

70 TH LD1 X,1(QLINK) n T7. Remove from queue. 
71 JMP 5B n F ¢ QLINK[F], go to T5. 
72 8H OUT BUFFER(TAPEOUT) 1 T8. End of process. 

73 IOC O(TAPEOUT) 1 Output last block and rewind. 
14 HLT 0,6 1 Stop, displaying N on console. 
75 X END TOPSORT Beginning of table area | 


The analysis of Algorithm T is quite simple with the aid of Kirchhoff’s law; 
the execution time has the approximate form cım + c2n, where m is the number 
of input relations, n is the number of objects, and cı and cg are constants. It 
is hard to imagine a faster algorithm for this problem! The exact quantities in 
the analysis are given with Program T above, where a = number of objects with 
no predecessor, b = number of tape records in input = [(m + 2)/50], and c = 
number of tape records in output = [(n + 1)/100]. Exclusive of input-output 
operations, the total running time in this case is only (832m+24n+7b+2c+ 16) u. 

A topological sorting technique similar to Algorithm T (but without the 
important feature of the queue links) was first published by A. B. Kahn, CACM 5 
(1962), 558-562. The fact that topological sorting of a partial ordering is always 
possible was first proved in print by E. Szpilrajn, Fundamenta Mathematica 16 
(1930), 386-389; he proved it for infinite sets as well as finite sets, and mentioned 
that the result was already known to several of his colleagues. 

In spite of the fact that Algorithm T is so efficient, we will study an even 
better algorithm for topological sorting in Section 7.4.1. 


EXERCISES 


> 1. [10] Operation (9) for popping up a stack mentions the possibility of UNDERFLOW; 
why doesn’t operation (8), pushing down a stack, mention the possibility of OVERFLOW? 
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2. [22] Write a “general purpose” MIX subroutine to do the insertion operation, (10). 
This subroutine should have the following specifications (as in Section 1.4.1): 


Calling sequence: JMP INSERT Jump to subroutine. 
NOP T Location of pointer variable 
Entry conditions: rA = information to be put into the INFO field of a new node. 
Exit conditions: The stack whose pointer is the link variable T has the new node on 
top; rll = T; rI2, rI8 are altered. 
3. [22] Write a “general purpose” MIX subroutine to do the deletion operation, (11). 
This subroutine should have the following specifications: 


Calling sequence: JMP DELETE Jump to subroutine. 
NOP T Location of pointer variable 
JMP UNDERFLOW First exit, if UNDERFLOW sensed 

Entry conditions: None 

Exit conditions: If the stack whose pointer is the link variable T is empty, the first 
exit is taken; otherwise the top node of that stack is deleted, and exit 
is made to the third location following ‘JMP DELETE’. In the latter 
case, rll = T and rA is the contents of the INFO field of the deleted 
node. In either case, rI2 and rI3 are used by this subroutine. 

4. [22] The program in (10) is based on the operation P < AVAIL, as given in 
(6). Show how to write an OVERFLOW subroutine so that, without any change in the 
coding (10), the operation P < AVAIL makes use of SEQMIN, as given by (7). For general- 
purpose use, your subroutine should not change the contents of any registers, except 
rJ and possibly the comparison indicator. It should exit to location rJ — 2, instead of 
the usual rJ. 


5. [24] Operations (14) and (17) give the effect of a queue; show how to define the 
further operation “insert at front” so as to obtain all the actions of an output-restricted 
deque. How could the operation “delete from rear” be defined (so that we would have 
a general deque)? 


6. [21] In operation (14) we set LINK(P) + A, while the very next insertion at the 
rear of the queue will change the value of this same link field. Show how the setting of 
LINK(P) in (14) could be avoided if we make a change to the testing of “F = A” in (17). 


7. [23] Design an algorithm to “invert” a linked linear list such as (1), that is, to 
change its links so that the items appear in the opposite order. [If, for example, the 
list (1) were inverted, we would have FIRST linking to the node containing item 5; that 
node would link to the one containing item 4; etc.] Assume that the nodes have the 
form (3). 


8. [24] Write a MIX program for the problem of exercise 7, attempting to design your 
program to operate as fast as possible. 


9. [20] Which of the following relations is a partial ordering on the specified set S? 
[Note: If the relation “x < y” is defined below, the intent is to define the relation 
“r ~ y = (a ~ yor x = y),” and then to determine whether < is a partial ordering.| 
(a) S = all rational numbers, x < y means x > y. (b) S = all people, x < y means 
x is an ancestor of y. (c) S = all integers, x < y means x is a multiple of y (that is, 
x mod y = 0). (d) S = all the mathematical results proved in this book, x < y means 
the proof of y depends upon the truth of x. (e) S = all positive integers, x < y means 
x + y is even. (f) S = a set of subroutines, x < y means “x calls y,” that is, y may be 
in operation while x is in operation, with recursion not allowed. 
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10. [M21] Given that “C” is a relation that satisfies properties (i) and (ii) of a partial 
ordering, prove that the relation “<”, defined by the rule “x < y if and only if z = y 
or x C y,” satisfies all three properties of a partial ordering. 


11. [24] The result of topological sorting is not always completely determined, since 
there may be several ways to arrange the nodes and to satisfy the conditions of 
topological order. Find all possible ways to arrange the nodes of Fig. 6 into topological 
order. 


12. [M20] There are 2” subsets of a set of n elements, and these subsets are partially 
ordered by the set-inclusion relation. Give two interesting ways to arrange these subsets 
in topological order. 


13. [M48] How many ways are there to arrange the 2” subsets described in exercise 12 
into topological order? (Give the answer as a function of n.) 


14. [M21] A linear ordering of a set S, also called a total ordering, is a partial ordering 
that satisfies the additional “comparability” condition 


(iv) For any two objects x, y in S, either x < y or y < z. 


Prove directly from the definitions given that a topological sort can result in only one 
possible output if and only if the relation < is a linear ordering. (You may assume that 
the set S is finite.) 


15. [M25] Show that for any partial ordering on a finite set S there is a unique set 
of irredundant relations that characterizes this ordering, as in (18) and Fig. 6. Is the 
same fact true also when S is an infinite set? 


16. [M22] Given any partial ordering on a set S = {x1,...,%n}, we can construct its 
incidence matrix (aij), where aij = 1 if a < xj, and aij = 0 otherwise. Show that 
there is a way to permute the rows and columns of this matrix so that all entries below 
the diagonal are zero. 


17. [21] What output does Algorithm T produce if it is presented with the input (18)? 


18. [20] What, if anything, is the significance of the values of QLINK[0], QLINK[1], 
.., QLINKEn] when Algorithm T terminates? 


19. [18] In Algorithm T we examine the front position of the queue in step T5, but 
do not remove that element from the queue until step T7. What would happen if we 
set F + QLINK[F] at the conclusion of step T5, instead of in T7? 


20. [24] Algorithm T uses F, R, and the QLINK table to obtain the effect of a queue that 
contains those nodes whose COUNT field has become zero but whose successor relations 
have not yet been removed. Could a stack be used for this purpose instead of a queue? 
If so, compare the resulting algorithm with Algorithm T. 


21. [21] Would Algorithm T still perform a valid topological sort if one of the relations 
“j < k” were repeated several times in the input? What if the input contained a relation 
of the form “j < j”? 


22. [23] Program T assumes that its input tape contains valid information, but a 
program that is intended for general use should always make careful tests on its input so 
that clerical errors can be detected, and so that the program cannot “destroy itself.” For 
example, if one of the input relations for k were negative, Program T may erroneously 
change one of its own instructions when storing into X[k]. Suggest ways to modify 
Program T so that it is suitable for general use. 
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> 23. [27] When the topological sort algorithm cannot proceed because it has detected 
a loop in the input (see step T8), it is usually of no use to stop and say, “There was 
a loop.” It is helpful to print out one of the loops, thereby showing part of the input 
that was in error. Extend Algorithm T so that it will do this additional printing of a 
loop when necessary. [Hint: The text gives a proof for the existence of a loop when 
N > 0 in step T8; that proof suggests an algorithm] 


24. [24] Incorporate the extensions of Algorithm T made in exercise 23 into Pro- 
gram T. 


25. [47] Design as efficient an algorithm as possible for doing a topological sort of very 
large sets S having considerably more nodes than the computer memory can contain. 
Assume that the input, output, and temporary working space are done with magnetic 
tape. [Possible hint: A conventional sort of the input allows us to assume that all 
relations for a given node appear together. But then what can be done? In particular, 
we must consider the worst case in which the given ordering is already a linear ordering 
that has been wildly permuted; exercise 24 in the introduction to Chapter 5 explains 
how to handle this case with O(log n)? passes over the data.] 


26. [29] (Subroutine allocation.) Suppose that we have a tape containing the main 
subroutine library in relocatable form, for a 1960s-style computer installation. The 
loading routine wants to determine the amount of relocation for each subroutine used, 
so that it can make one pass through the tape to load the necessary routines. The 
problem is that some subroutines require others to be present in memory. Infrequently 
used subroutines (which appear toward the end of the tape) may call on frequently 
used subroutines (which appear toward the beginning of the tape), and we want to 
know all of the subroutines that are required, before passing through the tape. 

One way to tackle this problem is to have a “tape directory” that fits in memory. 
The loading routine has access to two tables: 


a) The tape directory. This table is composed of variable-length nodes having 
the form 


B SPACE LINK B SPACE LINK 

B SUB1 SUB2 B SUB1 SUB2 
or 

B SUBn 0 B _|SUB(n—1)| SUBn 


where SPACE is the number of words of memory required by the subroutine; LINK is a 
link to the directory entry for the subroutine that appears on the tape following this 
subroutine; SUB1, SUB2, ..., SUBn (n > 0) are links to the directory entries for any 
other subroutines required by this one; B = 0 on all words except the last, B = —1 on 
the last word of a node. The address of the directory entry for the first subroutine on 
the library tape is specified by the link variable FIRST. 


b) The list of subroutines directly referred to by the program to be loaded. This is 
stored in consecutive locations X[1], X[2], ..., XIN], where N > 0 is a variable known 
to the loading routine. Each entry in this list is a link to the directory entry for the 
subroutine desired. 

The loading routine also knows MLOC, the amount of relocation to be used for the 
first subroutine loaded. 
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As a small example, consider the following configuration: 


Tape directory List of subroutines needed 

B SPACE LINK xX[1] = 1003 
1000: 0 20 1005 X[2] = 1010 
1001: —1 1002 0 
1002: —1 30 1010 N=2 
1003: 0 200 1007 FIRST = 1002 
1004: —1 1000 1006 MLOC = 2400 
1005: —1 100 1003 
1006: —1 60 1000 
1007: 0 200 0 
1008: 0 1005 1002 
1009: —1 1006 0 
1010: —1 20 1006 


The tape directory in this case shows that the subroutines on tape are 1002, 1010, 
1006, 1000, 1005, 1003, and 1007 in that order. Subroutine 1007 takes 200 locations 
and implies the use of subroutines 1005, 1002, and 1006; etc. The program to be loaded 
requires subroutines 1003 and 1010, which are to be placed into locations > 2400. These 
subroutines in turn imply that 1000, 1006, and 1002 must also be loaded. 

The subroutine allocator is to change the X-table so that each entry X[1], X [2], 
X[3], ... has the form 


+ | o | BASE | sUB 


(except the last entry, which is explained below), where SUB is a subroutine to be loaded, 
and BASE is the amount of relocation. These entries are to be in the order in which the 
subroutines appear on tape. One possible answer for the example above would be 


BASE SUB BASE SUB 
X[1]: 2400 1002 X[4]: 2510 1000 
X[2]: 2430 1010 X[5]: 2530 1003 
X[3]: 2450 1006 X[6]: 2730 0 


The last entry contains the first unused memory address. 

(Clearly, this is not the only way to treat a library of subroutines. The proper way 
to design a library is heavily dependent upon the computer used and the applications 
to be handled. Large modern computers require an entirely different approach to 
subroutine libraries. But this is a nice exercise anyway, because it involves interesting 
manipulations on both sequential and linked data.) 

The problem in this exercise is to design an algorithm for the stated task. Your 
allocator may transform the tape directory in any way as it prepares its answer, 
since the tape directory can be read in anew by the subroutine allocator on its next 
assignment, and the tape directory is not needed by other parts of the loading routine. 


27. [25] Write a MIX program for the subroutine allocation algorithm of exercise 26. 


28. [40] The following construction shows how to “solve” a fairly general type of two- 
person game, including chess, nim, and many simpler games: Consider a finite set of 
nodes, each of which represents a possible position in the game. For each position there 
are zero or more moves that transform that position into some other position. We say 
that position x is a predecessor of position y (and y is a successor of x) if there is a 
move from x to y. Certain positions that have no successors are classified as won or 
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lost positions. The player to move in position x is the opponent of the player to move 
in the successors of position x. 

Given such a configuration of positions, we can compute the complete set of won 
positions (those in which the next player to move can force a victory) and the complete 
set of lost positions (those in which the player must lose against an expert opponent) 
by repeatedly doing the following operation until it yields no change: Mark a position 
“lost” if all its successors are marked “won”; mark a position “won” if at least one of 
its successors is marked “lost.” 

After this operation has been repeated as many times as possible, there may be 
some positions that have not been marked at all; a player in such a position can neither 
force a victory nor be compelled to lose. 

This procedure for obtaining the complete set of won and lost positions can be 
adapted to an efficient algorithm for computers that closely resembles Algorithm T. 
We may keep with each position a count of the number of its successors that have not 
been marked “won,” and a list of all its predecessors. 

The problem in this exercise is to work out the details of the algorithm that has 

just been so vaguely described, and to apply it to some interesting games that do not 
involve too many possible positions [like the “military game”: É. Lucas, Récréations 
Mathématiques 3 (Paris: 1893) 105-116; E. R. Berlekamp, J. H. Conway, and R. K. 
Guy, Winning Ways 3 (A. K. Peters, 2003), Chapter 21]. 
29. [21] (a) Give an algorithm to “erase” an entire list like (1), by putting all of its 
nodes on the AVAIL stack, given only the value of FIRST. The algorithm should operate 
as fast as possible. (b) Repeat part (a) for a list like (12), given the values of F and R. 
30. [17] Suppose that queues are represented as in (12), but with an empty queue 
represented by F = A and R undefined. What insertion and deletion procedures should 
replace (14) and (17)? 


2.2.4. Circular Lists 


A slight change in the manner of linking furnishes us with an important alter- 
native to the methods of the preceding section. 

A circularly linked list (briefly: a circular list) has the property that its last 
node links back to the first instead of to A. It is then possible to access all of 
the list starting at any given point; we also achieve an extra degree of symmetry, 
and if we choose we need not think of the list as having a last or first node. 

The following situation is typical: 


+4 +H +H +H p K— PTR (1) 


Assume that the nodes have two fields, INFO and LINK, as in the preceding 
section. There is a link variable PTR that points to the rightmost node of the 
list, and LINK (PTR) is the address of the leftmost node. The following primitive 
operations are most important: 


a) Insert Y at left: P <= AVAIL, INFO(P) + Y, LINK(P) + LINK(PTR), 
LINK(PTR) < P. 

b) Insert Y at right: Insert Y at left, then PTR + P. 

c) Set Y to left node and delete: P + LINK(PTR), Y + INFO(P), LINK(PTR) + 
LINK(P), AVAIL < P. 
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Operation (b) is a little surprising at first glance; the operation PTR + LINK (PTR) 
effectively moves the leftmost node to the right in the diagram (1), and this is 
quite easy to understand if the list is regarded as a circle instead of as a straight 
line with connected ends. 

The alert reader will observe that we have made a serious mistake in oper- 
ations (a), (b), and (c). What is it? Answer: We have forgotten to consider the 
possibility of an empty list. If, for example, operation (c) is applied five times 
to the list (1), we will have PTR pointing to a node in the AVAIL list, and this 
can lead to serious difficulties; for example, imagine applying operation (c) once 
more! If we take the position that PTR will equal A in the case of an empty 
list, we could remedy the operations by inserting the additional instructions “if 
PTR = A, then PTR + LINK(P) + P; otherwise ...” after “INFO(P) + Y” in (a); 
preceding (c) by the test “if PTR = A, then UNDERFLOW”; and following (c) by “if 
PTR = P, then PTR + A” 

Notice that operations (a), (b), and (c) give us the actions of an output- 
restricted deque, in the sense of Section 2.2.1. Therefore we find in particular 
that a circular list can be used as either a stack or a queue. Operations (a) 
and (c) combined give us a stack; operations (b) and (c) give us a queue. These 
operations are only slightly less direct than their counterparts in the previous 
section, where we saw that operations (a), (b), and (c) can be performed on 
linear lists using two pointers F and R. 

Other important operations become efficient with circular lists. For example, 
it is very convenient to “erase” a list, that is, to put an entire circular list onto 
the AVAIL stack at once: 


If PTR Æ A, then AVAIL 4+ LINK(PTR). (2) 


[Recall that the “+” operation denotes interchange: P + AVAIL, AVAIL + 
LINK(PTR), LINK(PTR) < P.] Operation (2) is clearly valid if PTR points any- 
where in the circular list. Afterwards we should of course set PTR + A. 

Using a similar technique, if PTR; and PTR2 point to disjoint circular lists L4 
and Lg, respectively, we can insert the entire list Ly at the right of Lı: 


If PTR, Æ A, then 
(if PTR, Æ A, then LINK(PTR,) + LINK(PTR2); (3) 
set PTR; + PTR, PTR2 + A). 


Splitting one circular list into two, in various ways, is another simple opera- 
tion that can be done. These operations correspond to the concatenation and 
deconcatenation of strings. 

Thus we see that a circular list can be used not only to represent inherently 
circular structures, but also to represent linear structures; a circular list with one 
pointer to the rear node is essentially equivalent to a straight linear list with two 
pointers to the front and rear. The natural question to ask, in connection with 
this observation, is “How do we find the end of the list, when there is circular 
symmetry?” There is no A link to signal the end! The answer is that when we 
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are operating on an entire list, moving from one node to the next, we should stop 
when we get back to our starting place (assuming, of course, that the starting 
place is still present in the list). 

An alternative solution to the problem just posed is to put a special, recog- 
nizable node into each circular list, as a convenient stopping place. This special 
node is called the list head, and in applications we often find it is quite convenient 
to insist that every circular list must have exactly one node that is its list head. 
One advantage is that the circular list will then never be empty. With a list 
head, diagram (1) becomes 

List head 
ty [4> «| > by ag | ©) 


i 


References to lists like (4) are usually made via the list head, which is often in a 
fixed memory location. The disadvantage of list heads is that there is no pointer 
to the right end, so we must sacrifice operation (b) stated above. 

Diagram (4) may be compared with 2.2.3-(1) at the beginning of the previ- 
ous section, in which the link associated with “item 5” now points to LOC (FIRST) 
instead of to A; the variable FIRST is now thought of as a link within a node, 
namely the link that is in NODE(LOC(FIRST)). The principal difference between 
(4) and 2.2.3-(1) is that (4) makes it possible (though not necessarily efficient) 
to get to any point of the list from any other point. 


As an example of the use of circular lists, we will discuss arithmetic on 
polynomials in the variables x, y, and z, with integer coefficients. There are 
many problems in which a scientist wants to manipulate polynomials instead of 
just numbers; we are thinking of operations like the multiplication of 


(a4 + 22y + 3x2y? + day? + 5y*) by (a? — Qry + y’) 


to get 
(xê — Gary? + 5y°). 

Linked allocation is a natural tool for this purpose, since polynomials can grow to 
unpredictable sizes and we may want to represent many polynomials in memory 
at the same time. 

We will consider here the two operations of addition and multiplication. Let 
us suppose that a polynomial is represented as a list in which each node stands 
for one nonzero term, and has the two-word form 


COEF 
+| AÍ|B |C |LINK 


(5) 


Here COEF is the coefficient of the term in zêyBzC. We will assume that the 
coefficients and exponents will always lie in the range allowed by this format, and 
that it is not necessary to check the ranges during our calculations. The notation 
ABC will be used to stand for the + AB C fields of the node (5), treated as a single 
unit. The sign of ABC, namely the sign of the second word in (5), will always be 
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plus, except that there is a special node at the end of every polynomial that has 
ABC = —1 and COEF = 0. This special node is a great convenience, analogous to 
our discussion of a list head above, because it provides a convenient sentinel and 
it avoids the problem of an empty list (corresponding to the polynomial 0). The 
nodes of the list always appear in decreasing order of the ABC field, if we follow 
the direction of the links, except that the special node (which has ABC = —1) 
links to the largest value of ABC. For example, the polynomial zê — 6zy° + 5y® 
would be represented thus: PTR 


4 
+1 -6 +5 0 

(GEE HRAL Aek Sjg0ons: 

Algorithm A (Addition of polynomials). This algorithm adds polynomial (P) 

to polynomial(Q), assuming that P and Q are pointer variables pointing to 

polynomials having the form above. The list P will be unchanged; the list Q 

will retain the sum. Pointer variables P and Q return to their starting points 

at the conclusion of this algorithm; auxiliary pointer variables Q1 and Q2 are 
also used. 

A1. [Initialize.] Set P < LINK(P), Q1 + Q, Q + LINK(Q). (Now both P and Q 
point to the leading terms of their polynomials. Throughout most of this 
algorithm the variable Q1 will be one step behind Q, in the sense that Q = 
LINK (Q1).) 

A2. [ABC(P) : ABC(Q).] If ABC(P) < ABC(Q), set Q1 + Q and Q + LINK(Q) and 

repeat this step. If ABC(P) = ABC(Q), go to step A3. If ABC(P) > ABC(Q), 

go to step A5. 

A3. [Add coefficients.] (We’ve found terms with equal exponents.) If ABC(P) <0, 

the algorithm terminates. Otherwise set COEF(Q) + COEF(Q) + COEF (P). 

Now if COEF(Q) = 0, go to A4; otherwise, set P + LINK(P), Q1 + Q, 

Q «+ LINK(Q), and go to A2. (Curiously the latter operations are identical 

to step Al.) 

AA. [Delete zero term.] Set Q2 + Q, LINK(Q1) «+ Q + LINK(Q), and AVAIL = 

Q2. (A zero term created in step A3 has been removed from polynomial (Q).) 

Set P + LINK(P) and go back to A2. 

A5. [Insert new term.] (Polynomial(P) contains a term that is not present 
in polynomial(Q), so we insert it in polynomial(Q).) Set Q2 < AVAIL, 
COEF (Q2) < COEF (P), ABC(Q2) + ABC(P), LINK(Q2) + Q, LINK(Q1) + Q2, 
Qi «+ Q2, P + LINK(P), and return to step A2. I 


One of the most noteworthy features of Algorithm A is the manner in which 
the pointer variable Q1 follows the pointer Q around the list. This is very typical 
of list processing algorithms, and we will see a dozen more algorithms with the 
same characteristic. Can the reader see why this idea was used in Algorithm A? 

A reader who has little prior experience with linked lists will find it very 
instructive to study Algorithm A carefully; as a test case, try adding «+ y + z 
to z? — 2y — z. 
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Given Algorithm A, the multiplication operation is surprisingly easy: 


Algorithm M (Multiplication of polynomials). This algorithm, analogous to 
Algorithm A, replaces polynomial (Q) by 


polynomial(Q) + polynomial(M) x polynomial (P). 


M1. [Next multiplier.]} Set M << LINK(M). If ABC(M) < 0, the algorithm termi- 
nates. 


M2. [Multiply cycle.] Perform Algorithm A, except that wherever the notation 
“ABC(P)” appears in that algorithm, replace it by “(if ABC(P) < 0 then —1, 
otherwise ABC(P) + ABC(M))”; wherever “COEF(P)” appears in that algo- 
rithm replace it by “COEF(P) x COEF(M)”. Then go back to step M1. J 


The programming of Algorithm A in MIX language shows again the ease with 
which linked lists are manipulated in a computer. In the following code we 
assume that OVERFLOW is a subroutine that either terminates the program (due 
to lack of memory space) or finds further available space and exits to rJ — 2. 


Program A (Addition of polynomials). This is a subroutine written so that it 
can be used in conjunction with a multiplication subroutine (see exercise 15). 


Calling sequence: JMP ADD 

Entry conditions: rll = P, rI2 = Q. 

Exit conditions: polynomial(Q) has been replaced by polynomial(Q) + poly- 
nomial (P); rI1 and rI2 are unchanged; all other registers have 
undefined contents. 


In the coding below, P = rI1, Q = rI2, Q1 = rI3, and Q2 =rlI6, in the notation of 
Algorithm A. 


01 LINK EQU 4:5 Definition of LINK field 
02 ABC EQU 0:3 Definition of ABC field 
08 ADD STJ 3F 1 Entrance to subroutine 
04 1H ENT3 0,2 1+m” Al. Initialize. Set Q1 < Q. 
05 LD2 1,3(LINK) 1+ m” Q+¢ -LINK(Q1). 

06 OH LD1 1,1(LINK) 1+p P+ LINK(P). 

07 SW1 LDA 1,1 1+p  rA(0:3) + ABC(P). 

08 2H CMPA 1,2(ABC) x A2. ABC(P) : ABC(Q). 
09 JE 3F x If equal, go to A3. 

10 JG O5F p'+q If greater, go to A5. 

11 ENT3 0,2 q' If less, set Q1 + Q. 

12 LD2 1,3(LINK) q Q < LINK (Q1). 

13 JMP 2B q Repeat. 

14 3H JAN * m+1 A383. Add coefficients. 
15 SW2 LDA 0,1 m COEF (P) 

16 ADD 0,2 m + COEF (Q) 

17 STA 0,2 m — COEF (Q). 

18 JANZ 1B m Jump if nonzero. 
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19 ENT6 0,2 m A4. Delete zero term. Q2 + Q. 
20 LD2 1,2(LINK) m’ Q + LINK(Q). 

21 LDX AVAIL m’ 

22 STX 1,6(LINK) m' AVAIL = Q2. 

23 ST6 AVAIL m 

24 ST2 1,3(LINK) m LINK(Q1) «+ Q. 

25 JMP OB m’ Go to advance P. 

26 5H LD6 AVAIL p A5. Insert new term. 
27 J6Z OVERFLOW p 

28 LDX 1,6(LINK) p Re < AVAL; 

29 STX AVAIL p' 

30 STA 1,6 p ABC(Q2) + ABC(P). 

31 SW3 LDA 0,1 p' rA + COEF (P). 

32 STA 0,6 p COEF (Q2) + rA. 

33 ST2 1,6(LINK) p LINK (Q2) + Q. 

34 ST6 1,3(LINK) p LINK (Q1) ¢ Q2. 

35 ENT3 0,6 p Qi + Q2. 

36 JMP OB p Go to advance P. J 


Note that Algorithm A traverses each of the two lists just once; it is not 
necessary to loop around several times. Using Kirchhoff’s law, we find that an 
analysis of the instruction counts presents no difficulties; the execution time 
depends on four quantities 


m’ = number of matching terms that cancel with each other; 
m” = number of matching terms that do not cancel; 
p' = number of unmatched terms in polynomial (P); 
q! = number of unmatched terms in polynomial (Q). 


The analysis given with Program A uses the abbreviations 
m=m +m", p=mt+p, g=mt+q, c=l+mt+p'+d; 


the running time for MIX is (27m! + 18m” + 27p' + 8q'+13)u. The total number 
of nodes in the storage pool needed during the execution of the algorithm is at 
least 2 + p + q, and at most 2+p+q+p". 


EXERCISES 


1. [21] The text suggests at the beginning of this section that an empty circular list 
could be represented by PTR = A. It might be more consistent with the philosophy 
of circular lists to have PTR = LOC(PTR) indicate an empty list. Does this convention 
facilitate operations (a), (b), or (c) described at the beginning of this section? 


2. [20] Draw “before and after” diagrams illustrating the effect of the concatenation 
operation (3), assuming that PTR; and PTR2 are Æ A. 


3. [20] What does operation (3) do if PTR, and PTR2 are both pointing to nodes in 
the same circular list? 


4. [20] State insertion and deletion operations that give the effect of a stack, using 
representation (4). 
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A2. ABC(P) : ABC(Q) < 


A1. Initialize 


A3. Add A4. Delete A5. Insert 
coefficients zero term new term 


Sentinel 


Fig. 10. Addition of polynomials. 


> 5. [21] Design an algorithm that takes a circular list such as (1) and reverses the 
direction of all the arrows. 


6. [18] Give diagrams of the list representation for the polynomials (a) xz — 3; (b) 0. 


7. [10] Why is it useful to assume that the ABC fields of a polynomial list appear in 
decreasing order? 


> 8. [10] Why is it useful to have Q1 trailing one step behind Q in Algorithm A? 


> 9. [23] Would Algorithm A work properly if P = Q (i.e., both pointer variables point 
at the same polynomial)? Would Algorithm M work properly if P = M, if P = Q, or if 
M=Q? 

> 10. [20] The algorithms in this section assume that we are using three variables x, y, 
and z in the polynomials, and that their exponents individually never exceed b — 1 
(where b is the byte size in MIX’s case). Suppose instead that we want to do addition 
and multiplication of polynomials in only one variable, x, and to let its exponent take 
on values up to b? — 1. What changes should be made to Algorithms A and M? 


11. [24] (The purpose of this exercise and many of those following is to create a pack- 
age of subroutines useful for polynomial arithmetic, in conjunction with Program A.) 
Since Algorithms A and M change the value of polynomial (Q), it is sometimes desirable 
to have a subroutine that makes a copy of a given polynomial. Write a MIX subroutine 
with the following specifications: 


Calling sequence: JMP COPY 

Entry conditions: rll = P 

Exit conditions: rl2 points to a newly created polynomial equal to polynomial (P); 
rI1 is unchanged; other registers are undefined. 

12. [21] Compare the running time of the program in exercise 11 with that of Pro- 

gram A when polynomial(Q) = 0. 

13. [20] Write a MIX subroutine with the following specifications: 

Calling sequence: JMP ERASE 

Entry conditions: rll = P 

Exit conditions: polynomial(P) has been added to the AVAIL list; all register contents 
are undefined. 

[Note: This subroutine can be used in conjunction with the subroutine of exercise 11 


in the sequence ‘LD1 Q; JMP ERASE; LD1 P; JMP COPY; ST2 Q’ to achieve the effect 
“polynomial(Q) < polynomial (P)”.] 
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14. [22] Write a MIX subroutine with the following specifications: 


Calling sequence: JMP ZERO 

Entry conditions: None 

Exit conditions: rl2 points to a newly created polynomial equal to 0; other register 
contents are undefined. 

15. [24] Write a MIX subroutine to perform Algorithm M, having the following specifi- 

cations: 

Calling sequence: JMP MULT 

Entry conditions: rll = P, rl2=Q, rl4=M. 

Exit conditions: polynomial(Q) < polynomial(Q) + polynomial(M) x polynomial (P); 
rI1, rI2, rI4 are unchanged; other registers undefined. 


[Note: Use Program A as a subroutine, changing the settings of SW1, SW2, and SW3.] 
16. [M28] Estimate the running time of the subroutine in exercise 15 in terms of some 
relevant parameters. 

17. [22] What advantage is there in representing polynomials with a circular list as 
in this section, instead of with a straight linear linked list terminated by A as in the 
previous section? 


18. [25] Devise a way to represent circular lists inside a computer in such a way that 
the list can be traversed efficiently in both directions, yet only one link field is used 
per node. [Hint: If we are given two pointers, to two successive nodes x;~1 and 24, it 
should be possible to locate both xj; and x;_2.] 


2.2.5. Doubly Linked Lists 


For even greater flexibility in the manipulation of linear lists, we can include two 
links in each node, pointing to the items on either side of that node: 


we Og) TAT TATE Ty ks (1) 


Here LEFT and RIGHT are pointer variables to the left and right of the list. Each 
node of the list includes two links, called, for example, LLINK and RLINK. The 
operations of a general deque are readily performed with such a representation; 
see exercise 1. However, manipulations of doubly linked lists almost always 
become much easier if a list head node is part of each list, as described in the 
preceding section. When a list head is present, we have the following typical 
diagram of a doubly linked list: 


List head 


cc 
G) 
The RLINK and LLINK fields of the list head take the place of LEFT and RIGHT 
in (1). There is complete symmetry between left and right; the list head could 
equally well have been shown at the right of (2). If the list is empty, both link 
fields of the list head point to the head itself. 
The list representation (2) clearly satisfies the condition 


RLINK(LLINK(X)) = LLINK(RLINK(X)) = X (3) 
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if X is the location of any node in the list (including the head). This fact is the 
principal reason that representation (2) is preferable to (1). 

A doubly linked list usually takes more memory space than a singly linked 
one does (although there is sometimes room for another link in a node that 
doesn’t filla complete computer word). But the additional operations that can be 
performed efficiently with two-way links are often more than ample compensation 
for the extra space requirement. Besides the obvious advantage of being able to 
go back and forth at will when examining a doubly linked list, one of the principal 
new abilities is the fact that we can delete NODE(X) from the list it is in, given 
only the value of X. This deletion operation is easy to derive from a “before and 
after” diagram (Fig. 11) and it is very simple: 


RLINK (LLINK(X)) + RLINK (X), LLINK(RLINK(X)) + LLINK(X), 
AVAIL < X. 


(4) 


In a list that has only one-way links, we cannot delete NODE(X) without 
knowing which node precedes it in the chain, since the preceding node needs to 
have its link altered when NODE(X) is deleted. In all the algorithms considered in 
Sections 2.2.3 and 2.2.4 this additional knowledge was present whenever a node 
was to be deleted; see, in particular, Algorithm 2.2.4A, where we had pointer Q1 
following pointer Q for just this purpose. But we will meet several algorithms 
that require removing random nodes from the middle of a list, and doubly linked 
lists are frequently used just for this reason. (We should point out that in a 
circular list it is possible to delete NODE(X), given X, if we go around the entire 
circle to find the predecessor of X. But this operation is clearly inefficient when 
the list is long, so it is rarely an acceptable substitute for doubly linking the list. 
See also the answer to exercise 2.2.4-8.) 


| 


KN e RKO e | 
Before 
After TE | Fig. 11. Deletion from 
AVAIL a doubly linked list. 


Similarly, a doubly linked list permits the easy insertion of a node adjacent 
to NODE(X) at either the left or the right. The steps 


P < AVAIL, LLINK(P) X, RLINK(P) 4+ RLINK(X), w 
LLINK(RLINK(X)) 4+ P,  RLINK(X) P 5 


do such an insertion to the right of NODE(X); and by interchanging left and 
right we get the corresponding algorithm for insertion to the left. Operation 
(5) changes the settings of five links, so it is a little slower than an insertion 
operation in a one-way list where only three links need to be changed. 
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As an example of the use of doubly linked lists, we will now consider the 
writing of a discrete simulation program. “Discrete simulation” means the 
simulation of a system in which all changes in the state of the system may 
be assumed to happen at certain discrete instants of time. The “system” being 
simulated is usually a set of individual activities that are largely independent 
although they interact with each other; examples are customers at a store, ships 
in a harbor, people in a corporation. In a discrete simulation, we proceed by 
doing whatever is to be done at a certain instant of simulated time, then advance 
the simulated clock to the next time when some action is scheduled to occur. 

By contrast, a “continuous simulation” would be simulation of activities that 
are under continuous changes, such as traffic moving on a highway, spaceships 
traveling to other planets, etc. Continuous simulation can often be satisfactorily 
approximated by discrete simulation with very small time intervals between 
steps; however, in such a case we usually have “synchronous” discrete simulation, 
in which many parts of the system are slightly altered at each discrete time 
interval, and such an application generally calls for a somewhat different type of 
program organization than the kind considered here. 

The program developed below simulates the elevator system in the Mathe- 
matics building of the California Institute of Technology. The results of such a 
simulation will perhaps be of use only to people who make reasonably frequent 
visits to Caltech; and even for them, it may be simpler just to try using the 
elevator several times instead of writing a computer program. But, as is usual 
with simulation studies, the methods we will use are of much more interest than 
the answers given by the program. The methods to be discussed below illustrate 
typical implementation techniques used with discrete simulation programs. 

The Mathematics building has five floors: sub-basement, basement, first, 
second, and third. There is a single elevator, which has automatic controls 
and can stop at each floor. For convenience we will renumber the floors 0, 1, 
2, 3, and 4. 

On each floor there are two call buttons, one for UP and one for DOWN. 
(Actually floor 0 has only UP and floor 4 has only DOWN, but we may ignore 
that anomaly since the excess buttons will never be used.) Corresponding to 
these buttons, there are ten variables CALLUP[7] and CALLDOWN[7], 0 < j < 4. 
There are also variables CALLCAR[j], 0 < j < 4, representing buttons within 
the elevator car, which direct it to a destination floor. When a person presses a 
button, the appropriate variable is set to 1; the elevator clears the variable to 0 
after the request has been fulfilled. 

So far we have described the elevator from a user’s point of view; the 
situation is more interesting as viewed by the elevator. The elevator is in one of 
three states: GOINGUP, GOINGDOWN, or NEUTRAL. (The current state is indicated 
to passengers by lighted arrows inside the elevator.) If it is in NEUTRAL state and 
not on floor 2, the machine will close its doors and (if no command is given by 
the time its doors are shut) it will change to GOINGUP or GOINGDOWN, heading 
for floor 2. (This is the “home floor,” since most passengers get in there.) On 
floor 2 in NEUTRAL state, the doors will eventually close and the machine will wait 
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silently for another command. The first command received for another floor 
sets the machine GOINGUP or GOINGDOWN as appropriate; it stays in this state 
until there are no commands waiting in the same direction, and then it switches 
direction or switches to NEUTRAL just before opening the doors, depending on 
what other commands are in the CALL variables. The elevator takes a certain 
amount of time to open and close its doors, to accelerate and decelerate, and 
to get from one floor to another. All of these quantities are indicated in the 
algorithm below, which is much more precise than an informal description can 
be. The algorithm we will now study may not reflect the elevator’s true principles 
of operation, but it is believed to be the simplest set of rules that explain all 
the phenomena observed during several hours of experimentation by the author 
during the writing of this section. 

The elevator system is simulated by using two coroutines, one for the pas- 
sengers and one for the elevator; these routines specify all the actions to be 
performed, as well as various time delays that are to be used in the simulation. 
In the following description, the variable TIME represents the current value of 
the simulated time clock. All units of time are given in tenths of seconds. There 
are also several other variables: 


FLOOR, the current position of the elevator; 

D1, a variable that is zero except during the time people are getting in or 
out of the elevator; 

D2, a variable that becomes zero if the elevator has sat on one floor without 
moving for 30 sec or more; 

D3, a variable that is zero except when the doors are open but nobody is 
getting in or out of the elevator; 

STATE, the current state of the elevator (GOINGUP, GOINGDOWN, or NEUTRAL). 


Initially FLOOR = 2, D1 = D2 = D3 = 0, and STATE = NEUTRAL. 


Coroutine U (Users). Everyone who enters the system begins to perform the 

actions specified below, starting at step U1. 

U1. [Enter, prepare for successor.] The following quantities are determined in 
some manner that will not be specified here: 


IN, the floor on which the new user has entered the system; 

OUT, the floor to which this user wants to go (OUT Æ IN); 

GIVEUPTIME, the amount of time this user will wait for the elevator before 
running out of patience and deciding to walk; 

INTERTIME, the amount of time before another user will enter the system. 


After these quantities have been computed, the simulation program sets 
things up so that another user enters the system at TIME + INTERTIME. 


U2. [Signal and wait.] (The purpose of this step is to call for the elevator; some 
special cases arise if the elevator is already on the right floor.) If FLOOR = IN 
and if the elevator’s next action is step E6 below (that is, if the elevator doors 
are now closing), send the elevator immediately to its step E3 and cancel its 
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U3. 


U4. 


U5. 


U6. 
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activity E6. (This means that the doors will open again before the elevator 
moves.) If FLOOR = IN and if D3 Æ 0, set D3 + 0, set D1 to a nonzero value, 
and start up the elevator’s activity E4 again. (This means that the elevator 
doors are open on this floor, but everyone else has already gotten on or 
off. Elevator step E4 is a sequencing step that grants people permission to 
enter the elevator according to normal laws of courtesy; therefore, restarting 
E4 gives this user a chance to get in before the doors close.) In all other 
cases, the user sets CALLUP [IN] + 1 or CALLDOWN[IN] + 1, according as 
OUT > IN or OUT < IN; and if D2 = 0 or the elevator is in its “dormant” 
position E1, the DECISION subroutine specified below is performed. (The 
DECISION subroutine is used to take the elevator out of NEUTRAL state at 
certain critical times.) 


[Enter queue.] Insert this user at the rear of QUEUE [IN], which is a linear 
list representing the people waiting on this floor. Now the user waits 
patiently for GIVEUPTIME units of time, unless the elevator arrives first — 
more precisely, unless step E4 of the elevator routine below sends this user 
to U5 and cancels the scheduled activity U4. 


[Give up.] If FLOOR Æ IN or D1 = 0, delete this user from QUEUE [IN] 
and from the simulated system. (The user has decided that the elevator is 
too slow, or that a bit of exercise will be better than an elevator ride.) If 
FLOOR = IN and D1 Æ 0, the user stays and waits (knowing that the wait 
won't be long). 


[Get in.] This user now leaves QUEUE[IN] and enters ELEVATOR, which is 
a stack-like list representing the people now on board the elevator. Set 
CALLCAR [OUT] + 1. 

Now if STATE = NEUTRAL, set STATE < GOINGUP or GOINGDOWN as 
appropriate, and set the elevator’s activity E5 to be executed after 25 units 
of time. (This is a special feature of the elevator, allowing the doors to close 
faster than usual if the elevator is in NEUTRAL state when the user selects a 
destination floor. The 25-unit time interval gives step E4 the opportunity 
to make sure that D1 is properly set up by the time step E5, the door-closing 
action, occurs.) 

Now the user waits until being sent to step U6 by step E4 below, when 
the elevator has reached the desired floor. 


[Get out.] Delete this user from the ELEVATOR list and from the simulated 
system. J 


Coroutine E (Elevator). This coroutine represents the actions of the elevator; 
step E4 also handles the control of when people get in and out. 


El. 


E2. 


[Wait for call.] (At this point the elevator is sitting at floor 2 with the doors 
closed, waiting for something to happen.) If someone presses a button, the 
DECISION subroutine will take us to step E3 or E6. Meanwhile, wait. 

[Change of state?] If STATE = GOINGUP and CALLUP[j] = CALLDOWN[j] = 
CALLCAR[j] = 0 for all j > FLOOR, then set STATE 4+ NEUTRAL or STATE + 
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E3. 


E4. 


E5. 


E6. 


E7. 


E8. 


E9. 


GOINGDOWN, according as CALLCAR[j] = 0 for all 7 < FLOOR or not, and set 
all CALL variables for the current floor to zero. If STATE = GOINGDOWN, do 
similar actions with directions reversed. 


[Open doors.] Set D1 and D2 to any nonzero values. Set elevator activity 
E9 to start up independently after 300 units of time. (This activity may be 
canceled in step E6 below before it occurs. If it has already been scheduled 
and not canceled, we cancel it and reschedule it.) Also set elevator activity 
E5 to start up independently after 76 units of time. Then wait 20 units of 
time (to simulate opening of the doors) and go to E4. 


[Let people out, in.] If anyone in the ELEVATOR list has OUT = FLOOR, send 
the user of this type who has most recently entered immediately to step U6, 
wait 25 units, and repeat step E4. If no such users exist, but QUEUE [FLOOR] 
is not empty, send the front person of that queue immediately to step U5 
instead of U4, wait 25 units, and repeat step E4. But if QUEUE[FLOOR] 
is empty, set D1 + 0, make D3 nonzero, and wait for some other activity 
to initiate further action. (Step E5 will send us to E6, or step U2 will 
restart E4.) 

[Close doors.] If D1 4 0, wait 40 units and repeat this step (the doors flutter 
a little, but they spring open again, since someone is still getting out or in). 
Otherwise set D3 + 0 and set the elevator to start at step E6 after 20 units 
of time. (This simulates closing the doors after people have finished getting 
in or out; but if a new user enters on this floor while the doors are closing, 
they will open again as stated in step U2.) 


[Prepare to move.] Set CALLCAR[FLOOR] to zero; also set CALLUP [FLOOR] 
to zero if STATE Æ GOINGDOWN, and also set CALLDOWN [FLOOR] to zero if 
STATE # GOINGUP. (Note: If STATE = GOINGUP, the elevator does not clear 
out CALLDOWN, since it assumes that people who are going down will not 
have entered; but see exercise 6.) Now perform the DECISION subroutine. 

If STATE = NEUTRAL even after the DECISION subroutine has acted, go 
to El. Otherwise, if D2 4 0, cancel the elevator activity E9. Finally, if 
STATE = GOINGUP, wait 15 units of time (for the elevator to build up speed) 
and go to E7; if STATE = GOINGDOWN, wait 15 units and go to E8. 


[Go up a floor.] Set FLOOR + FLOOR + 1 and wait 51 units of time. If 
now CALLCAR[FLOOR] = 1 or CALLUP[FLOOR] = 1, or if ((FLOOR = 2 or 
CALLDOWN [FLOOR] = 1) and CALLUP[j] = CALLDOWN[j] = CALLCAR[j] = 0 
for all j > FLOOR), wait 14 units (for deceleration) and go to E2. Otherwise, 
repeat this step. 

[Go down a floor.] This step is like E7 with directions reversed, and also 
the times 51 and 14 are changed to 61 and 23, respectively. (It takes the 
elevator longer to go down than up.) 

[Set inaction indicator.] Set D2 + 0 and perform the DECISION subroutine. 
(This independent action is initiated in step E3 but it is almost always 
canceled in step E6. See exercise 4.) J 


Table 1 
SOME ACTIONS OF THE ELEVATOR SYSTEM 


987 


TIME STATE FLOOR Di D2 D3 step action TIME STATE FLOOR Di D2 D3 step action 5 
0000 N 2 0 0 0 U1 User 1 arrives at floor 0, destination is 2. 083 D 1 X X 0 U6 User 4 gets out, leaves the system. rj 
0035 D 2 0 0 0 E8 Elevator moving down 108 D X X 0 U6 User 3 gets out, leaves the system. © 
0038 D 1 0 0 0 U1 User 2 arrives at floor 4, destination is 1. 133 D X X 0 U6 User 5 gets out, leaves the system. z 
0096 D 1 0 0 0 E8 Elevator moving down 139 D X X 0 E5 Doors flutter. > 
0136 D 0 0 0 0 U1 User 3 arrives at floor 2, destination is 1. 158 D X X 0 U6 User 2 gets out, leaves the system. S] 
0141 D 0 0 0 0 Ul User 4 arrives at floor 2, destination is 1. 179 D X X 0 E5 Doors flutter. re) 
0152 D 0 0 0 0 U4 User 1 decides to give up, leaves the system. 183 D 1 X X 0 U5 User 7 gets in. Z 
0180 D 0 0 0 0 E2 Elevator stops. 208 D 1 X X 0 U5 User 8 gets in. 

0180 N 0 0 xX 0 E3 Elevator doors start to open. 219 D xX X 0 E5 Doors flutter. 4 
0200 N 0 X X 0 E4 Doors open, nobody is there. 233 D X X 0 U5 User 9 gets in. zo) 
0256 N 0 0 X X E5 Elevator doors start to close. 259 D 0 X X E5 Elevator doors start to close. (= 
0291 U 0 0 X 0 Ul User 5 arrives at floor 3, destination is 1. 294 D 0 X 0 E8 Elevator moving down Q 
0291 U 0 0 X 0 E7 Elevator moving up 378 D 0 0 X 0 E2 Elevator stops. a 
0342 U 1 0 X 0 E7 Elevator moving up 378 U 0 0 X 0 E3 Elevator doors start to open. fzo] 
0364 U 2 0 X 0 Ul User 6 arrives at floor 2, destination is 1. 398 U 0 X X 0 U6 User 8 gets out, leaves the system. les] 
0393 U 2 0 X 0 E7 Elevator moving up 423 U 0 x $ 0 U5 User 10 gets in. un 
0444 U 3 0 X 0 E7 Elevator moving up 454 U 0 0 X X E5 Elevator doors start to close. 

0509 U 4 0 X 0 E2 Elevator stops. 489 U 0 0 X 0 E7 Elevator moving u 

0509 N 4 0 xX 0 E3 Elevator doors start to open. 554 U 1 0 xX 0 E2 Elevator stops. 

0529 N 4 X X 0 U5 User 2 gets in. 554 U 1 0 X 0 E3 Elevator doors start to open. 

0540 D 4 X X 0 U4 User6 decides to give up, leaves the system. 630 U 1 0 X X E5 Elevator doors start to close. 

0554 D 4 0 X X E5 Elevator doors start to close. 665 U 1 0 X 0 E7 Elevator moving u 

0589 D 4 0 X 0 E8 Elevator moving down tee 

0602 D 3 0 X 0 U1 User 7 arrives at floor 1, destination is 2. 4257 N 2 0 xX 0 EL Elevator dormant 

0673 D 3 0 xX 0 E2 Elevator stops. 4384 N 2 0 xX 0 Ul User 17 arrives at floor 2, destination is 3. 

0673 D 3 0 X 0 E3 Elevator doors start to open. 4404 N 2 0 X 0 E3 Elevator doors start to open. 

0693 D 3 X X 0 U5 User 5 gets in. 4424 N 2 xX X 0 U5 User 17 gets in. 

0749 D 3 0 X X E5 Elevator doors start to close. 4449 U 2 0 X X E5 Elevator doors start to close. 

0784 D 3 0 X 0 E8 Elevator moving down 4484 U 2 0 X 0 E7 Elevator moving u 

0827 D 2 0 xX 0 Ul User 8 arrives at floor 1, destination is 0. 4549 U 3 0 xX 0 E2 Elevator stops. 

0868 D 2 0 -X 0 E2 Elevator stops. 4549 N 3 0 X 0 E3 Elevator doors start to open. 

0868 D 2 0 X 0 E3 Elevator doors start to open. 4569 N 3 > X 0 U6 User 17 gets out, leaves the system. 

0876 D 2 X X 0 Ul User 9 arrives at floor 1, destination is 3. 4625 N 3 0 X X E5 Elevator doors start to close. 

0888 D 2 X X 0 U5 User3 gets in. 4660 D 3 0 X 0 E8 Elevator moving down 

0913 D 2 X X 0 U5 User 4 gets in. 4744 D 2 0 xX 0 E2 Elevator stops. 

0944 D 2 0 X X E5 Elevator doors start to close. 4744 N 2 0X 0 E3 Elevator doors start to open. 

0979 D 2 0 X 0 E8 Elevator moving down 4764 N 2 X X 0 E4 Doors open, nobody is there. 

1048 D 1 0 X 0 Ul User 10 arrives at floor 0, destination is 4. 4820 N 2 0 X 0 E5 Elevator doors start to close. 

1063 D i 0 X 0 E2 Elevator stops. 4840 N 2 0 xX 0 El Elevator dormant ~y 
1063 D 1 0 X 0 E3 Elevator doors start to open. tee > 
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Subroutine D (DECISION subroutine). This subroutine is performed at certain 
critical times, as specified in the coroutines above, when a decision about the 
elevator’s next direction is to be made. 


D1. [Decision necessary?] If STATE # NEUTRAL, exit from this subroutine. 


D2. [Should doors open?] If the elevator is positioned at E1 and if CALLUP [2], 
CALLCAR [2], and CALLDOWN[2] are not all zero, cause the elevator to start 
its activity E3 after 20 units of time, and exit from this subroutine. (If 
the DECISION subroutine is currently being invoked by the independent 
activity E9, it is possible for the elevator coroutine to be positioned at E1.) 


D3. [Any calls?] Find the smallest j 4 FLOOR for which CALLUP[j], CALLCARLJ], 
or CALLDOWN[j] is nonzero, and go on to step D4. But if no such j exists, 
then set j < 2 if the DECISION subroutine is currently being invoked by 
step E6; otherwise exit from this subroutine. 


D4. [Set STATE.] If FLOOR > j, set STATE + GOINGDOWN; if FLOOR < j, set 
STATE < GOINGUP. 


D5. [Elevator dormant?) If the elevator coroutine is positioned at step E1, and 
if j Æ 2, set the elevator to perform step E6 after 20 units of time. Exit 
from the subroutine. J 


The elevator system described above is quite complicated by comparison 
with other algorithms we have seen in this book, but the choice of a real-life 
system is more typical of a simulation problem than any cooked-up “textbook 
example” would ever be. 

To help understand the system, consider Table 1, which gives part of the 
history of one simulation. It is perhaps best to start by examining the simple 
case starting at time 4257: The elevator is sitting idly at floor 2 with its doors 
shut, when a user arrives (time 4384); let’s say the user’s name is Don. Two 
seconds later, the doors open, and Don gets in after two more seconds. By 
pushing button “3” he starts the elevator moving up; ultimately he gets off at 
floor 3 and the elevator returns to floor 2. 

The first entries in Table 1 show a much more dramatic scenario: A user 
calls the elevator to floor 0, but loses patience and gives up after 15.2 sec. The 
elevator stops at floor 0 but finds nobody there; then it heads to floor 4, since 
there are several calls wanting to go downward; etc. 

The programming of this system for a computer (in our case, MIX) merits 
careful study. At any given time during the simulation, we may have many 
simulated users in the system (in various queues and ready to “give up” at various 
times), and there is also the possibility of essentially simultaneous execution of 
steps E4, E5, and E9 if many people are trying to get out as the elevator is 
trying to close its doors. The passing of simulated time and the handling of 
“simultaneity” may be programmed by having each entity represented by a node 
that includes a NEXTTIME field (denoting the time when the next action for this 
entity is to take place) and a NEXTINST field (denoting the memory address where 
this entity is to start executing instructions, analogous to ordinary coroutine 
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Fig. 12. Some lists used in the elevator simulation program. (List heads appear at 
the left.) 


linkage). Each entity waiting for time to pass is placed in a doubly linked 
list called the WAIT list; this “agenda” is sorted on the NEXTTIME fields of its 
nodes, so that the actions may be processed in the correct sequence of simulated 
times. The program also uses doubly linked lists for the ELEVATOR and for the 
QUEUE lists. 

Each node representing an activity (whether a user or an elevator action) 
has the form 


+ IN LLINK1 RLINK1 
+ NEXTTIME 
(6) 
+ NEXTINST 0 0 39 
+ OUT LLINK2 RLINK2 


Here LLINK1 and RLINK1 are the links for the WAIT list; LLINK2 and RLINK2 are 
used as links in the QUEUE lists or the ELEVATOR. The latter two fields and the 
IN and OUT field are relevant when node (6) represents a user, but they are not 
relevant for nodes that represent elevator actions. The third word of the node is 
actually a MIX ‘JMP’ instruction. 

Figure 12 shows typical contents of the WAIT list, ELEVATOR list, and one of 
the QUEUE lists; each node in the QUEUE list is simultaneously in the WAIT list 
with NEXTINST = U4, but this has not been indicated in the figure, since the 
complexity of the linking would obscure the basic idea. 

Now let us consider the program itself. It is quite long, although (as with all 
long programs) it divides into small parts each of which is quite simple in itself. 
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First comes a number of lines of code that just serve to define the initial contents 
of the tables. There are several points of interest here: We have list heads for the 
WAIT list (lines 010-011), the QUEUE lists (lines 026-031), and the ELEVATOR list 
(lines 032-033). Each of them is a node of the form (6), but with unimportant 
words deleted; the WAIT list head contains only the first two words of a node, 
and the QUEUE and ELEVATOR list heads require only the last word of a node. 
We also have four nodes that are always present in the system (lines 012-023): 
USER1, a node that is always positioned at step U1 ready to enter a new user 
into the system; ELEV1, a node that governs the main actions of the elevator 
at steps El, E2, E3, E4, E6, E7, and E8; and ELEV2 and ELEV3, nodes that 
are used for the elevator actions E5 and E9, which take place independently of 
other elevator actions with respect to simulated time. Each of these four nodes 
contains only three words, since they never appear in the QUEUE or ELEVATOR 
lists. The nodes representing each actual user in the system will appear in a 
storage pool following the main program. 


001 * THE ELEVATOR SIMULATION 

002 IN EQU 1:1 Definition of fields 

003 LLINKi EQU 2:3 within nodes 

004 RLINK1 EQU 4:5 

005 NEXTINST EQU 0:2 

006 OUT EQU 1:1 

007 LLINK2 EQU 2:3 

008 RLINK2 EQU 4:5 

009 * FIXED-SIZE TABLES AND LIST HEADS 

010 WAIT CON ¥*+2(LLINK1) ,*+2(RLINK1) List head for WAIT list 

011 CON O NEXTTIME = 0 always 
012 USER1 CON ¥*-2(LLINK1) ,*-2(RLINK1) This node represents action 
013 CON 0O U1 and it is initially the 
014 JMP U1 sole entry in the WAIT list. 
015 ELEV1 CON 0 This node represents the 
016 CON O elevator actions, except 
017 JMP El for E5 and E9. 

018 ELEV2 CON 0 This node represents the 
019 CON 0 independent elevator 
020 JMP E5 action at E5. 

021 ELEV3 CON 0 This node represents the 
022 CON 0 independent elevator 
023 JMP E9 action at E9. 

024 AVAIL CON 0 Link to available nodes 

025 TIME CON 0 Current simulated time 
026 QUEUE EQU *-3 

027 CON ¥*-3(LLINK2) ,*-3(RLINK2) List head for QUEUE [0] 

028 CON ¥*-3(LLINK2) ,*-3(RLINK2) List head for QUEUE [1] 

029 CON ¥*-3(LLINK2) ,*-3(RLINK2) All queues initially 

030 CON ¥*-3(LLINK2) ,*-3(RLINK2) are empty 

031 CON ¥*-3(LLINK2) ,*-3(RLINK2) List head for QUEUE [4] 
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032 ELEVATOR EQU *-3 


033 CON *-3(LLINK2),¿,*-3(RLINK2) List head for ELEVATOR 

034 CON Oo 

035 CON 0O “Padding” for CALL table 

036 CON 0O (see lines 183-186) 

037 CON O 

038 CALL CON O CALLUP [0], CALLCAR [0], CALLDOWN [0] 
039 CON o0 CALLUP[1], CALLCAR[1], CALLDOWN [1] 
O40 CON Oo CALLUP [2], CALLCAR[2], CALLDOWN [2] 
041 CON Oo CALLUP [3], CALLCAR[3], CALLDOWN [3] 
O42 CON Oo CALLUP [4], CALLCAR[4], CALLDOWN [4] 
043 CON o0 

O44 CON O “Padding” for CALL table 

O45 CON 0 (see lines 178-181) 

046 CON Oo 

O47 D1 CON 0 Indicates doors open, activity 
048 D2 CON O Indicates no prolonged standstill 
049 D3 CON 0 Indicates doors open, inactivity | 


The next part of the program coding contains basic subroutines and the 
main control routines for the simulation process. Subroutines INSERT and DELETE 
perform typical manipulations on doubly linked lists; they put the current node 
into or take it out of a QUEUE or ELEVATOR list. (In the program, the “current 
node” C is always represented by index register 6.) There are also subroutines for 
the WAIT list: Subroutine SORTIN adds the current node to the WAIT list, sorting 
it into the right place based on its NEXTTIME field. Subroutine IMMED inserts the 
current node at the front of the WAIT list. Subroutine HOLD puts the current node 
into the WAIT list, with NEXTTIME equal to the current time plus the amount in 
register A. Subroutine DELETEW deletes the current node from the WAIT list. 

The routine CYCLE is the heart of the simulation control: It decides which 
activity is to act next (namely, the first element of the WAIT list, which we know 
is nonempty), and jumps to it. There are two special entrances to CYCLE: CYCLE1 
first sets NEXTINST in the current node, and HOLDC is the same with an additional 
call on the HOLD subroutine. Thus, the effect of the instruction ‘JMP HOLDC’ with 
amount t in register A is to suspend activity for t units of simulated time and 
then to return to the following location. 


050 * SUBROUTINES AND CONTROL ROUTINE 


051 INSERT STJ OF Insert NODE(C) to left of NODE(rI1): 
052 LD2 3,1(LLINK2) rl2 + LLINK2(rI1). 

058 ST2 3,6(LLINK2) LLINK2(C) < rl2. 

054 ST6 3,1(LLINK2) LLINK2(rI1) + C. 

055 ST6 3,2(RLINK2) RLINK2(rI2) + C. 

056 ST1 3,6(RLINK2) RLINK2(C) < rll. 

057 9H JMP * Exit from subroutine. 

058 DELETE STJ 9F Delete NODE(C) from its list: 

059 LD1 3,6(LLINK2) P «+ LLINK2(C). 


060 LD2 3,6(RLINK2) Q + RLINK2(C). 
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061 
062 
068 
064 
065 
066 
067 
068 
069 
070 
O71 
O72 
O78 
O74 
O75 
O76 
O77 
O78 
079 
080 
081 
082 
083 
084 
085 
086 
087 
088 
089 
090 
091 
092 
093 
O94 
095 
096 


9H 
IMMED 


HOLD 
SORTIN 


2H 


9H 
DELETEW 


9H 
CYCLE1 


HOLDC 


CYCLE 


ST1 
ST2 
JMP 
STJ 
LDA 
STA 
ENT1 
JMP 
ADD 
STJ 
STA 
ENT1 
LD1 
CMPA 
JL 
LD2 
ST2 
ST1 
ST6 
ST6 
JMP 
STJ 
LD1 
LD2 
ST1 
ST2 
JMP 
STJ 
JMP 
STJ 
JMP 
LD6 
LDA 
STA 
JMP 
JMP 


3,2(LLINK2) 
3,1(RLINK2) 


OF 
1,6 

WAIT 

0, 1(LLINK1) 
1,1 

*-2 

0, 1(RLINK1) 
0,6(RLINK1) 
0,6(LLINK1) 
0, 1(RLINK1) 
0,2(LLINK1) 

* 

OF 
0,6(LLINK1) 
0,6(RLINK1) 
0,2(LLINK1) 
0,1 (RLINK1) 

* 
2,6(NEXTINST) 
CYCLE 
2,6(NEXTINST) 
HOLD 

WAIT (RLINK1) 
1,6 

TIME 

DELETEW 

2,6 


DOUBLY LINKED LISTS 


LLINK2(Q) +P. 

RLINK2(P) Q. 

Exit from subroutine. 

Insert NODE(C) first in WAIT list: 


Set NEXTTIME(C) <+ TIME. 

P< LOC(WAIT). 

Insert NODE(C) to right of NODE(P). 

rA + TIME + rA. 

Sort NODE (C) into WAIT list: 

Set NEXTTIME(C) + rA. 

P «+ LOC (WAIT). 

P «+ LLINK1 (P). 

Compare NEXTTIME fields, right to left. 

Repeat until NEXTTIME(C) > NEXTTIME (P). 

Q + RLINK1 (P). 

RLINK1(C) < Q. 

LLINK1(C) +P. 

RLINK1(P) =C. 

LLINK1(Q) + C. 

Exit from subroutine. 

Delete NODE(C) from WAIT list: 

(This is same as lines 058-063 
except LLINK1, RLINK1 are used 
instead of LLINK2, RLINK2.) 


Set NEXTINST(C) + rJ. 


Set NEXTINST(C) + rJ. 
Insert NODE(C) in WAIT, delay rA. 
Set current node C + RLINK1(LOC(WAIT)). 


TIME «+ NEXTTIME(C). 
Remove NODE(C) from WAIT list. 
Jump to NEXTINST(C). Jf 
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Now comes the program for Coroutine U. At the beginning of step U1, 
the current node C is USER1 (see lines 012-014 above), and lines 099-100 of the 
program cause USER1 to be reinserted into the WAIT list so that the next user 
will be generated after INTERTIME units of simulated time. The following lines 
101-114 take care of setting up a node for the newly generated user; the IN and 
OUT floors are recorded in this node position. The AVAIL stack is singly linked 
in the RLINK1 field of each node. Note that lines 101-108 perform the action 
“C < AVAIL” using the POOLMAX technique, 2.2.3-(7); no test for OVERFLOW is 
necessary here, since the total size of the storage pool (the number of users in 
the system at any one time) rarely exceeds 10 nodes (40 words). The return of 
a node to the AVAIL stack appears in lines 156-158. 
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Throughout the program, index register 4 equals the variable FLOOR, and 


index register 5 is positive, negative, or zero, depending on whether STATE = 
GOINGUP, GOINGDOWN, or NEUTRAL. The variables CALLUP [j], CALLCAR[j], and 
CALLDOWN[7] occupy the respective fields (1:1), (3:3), and (5:5) of location 


CALL + j. 


097 
098 
099 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 
132 
133 
134 
135 
136 
137 
138 
139 


* COROUTINE U 


U1 


1H 


U2 


3H 


4H 


2H 


JMP 
LDA 
JMP 
LD6 
J6P 
LD6 
INC6 
ST6 
JMP 
LDA 
STA 
LD1 
ST1 
LD2 
ST2 
ENTA 
STA 
ENTA 
DECA 
ST6 
JANZ 
ENT6 
LDA 
DECA 
JANZ 
ENTA 
STA 
JMP 
JMP 
LDA 
JAZ 
ST6 
STZ 
JMP 
JMP 
DEC2 
ENTA 
J2P 
STA 
JMP 
STA 
LDA 


VALUES 
INTERTIME 
HOLD 

AVAIL 

1F 

POOLMAX (0:2) 
4 

POOLMAX (0:2) 
*4+3 
0,6(RLINK1) 
AVAIL 
INFLOOR 
0,6(IN) 
OUTFLOOR 

3,6 (OUT) 

39 

2,6 

0,4 

0,1 

TEMP 

QF 

ELEV1 
2,6(NEXTINST) 
E6 

3F 

E3 
2,6(NEXTINST) 
DELETEW 

4F 


*+3 
CALL, 1(5:5) 
*+2 
CALL, 1(1:1) 
D2 


U1. Enter, prepare for successor. 

Set INFLOOR, OUTFLOOR, GIVEUPTIME, INTERTIME. 
INTERTIME is computed by VALUES subroutine. 
Put NODE(C) in WAIT, delay INTERTIME. 

C + AVAIL. 

If AVAIL Æ A, jump. 


C + POOLMAX + 4. 
POOLMAX & C. 
Assume that memory overflow won’t happen. 


AVAIL + RLINK1 (AVAIL). 

rll + INFLOOR (computed by VALUES above). 

IN(C) + rIl. 

rl2 < OUTFLOOR (computed by VALUES). 

OUT (C) + r12. 

Put constant 39 (JMP operation code) 
into third word of node format (6). 

U2. Signal and wait. Set rA < FLOOR. 

FLOOR — IN. 

Save value of C. 

Jump if FLOOR Æ IN. 

Set C + LOC(ELEV1). 

Is elevator positioned at E6? 


If so, reposition it at E3. 


Remove it from WAIT list 
and reinsert it at front of WAIT. 


Jump if D3 = 0. 

Otherwise make D1 nonzero. 

Set D3 + 0. 

Insert ELEV1 at front of WAIT list. 
(rll and rI2 have changed.) 

rl2 + OUT — IN. 


Jump if going up. 
Set CALLDOWN [IN] + 1. 


Set CALLUP[IN] + 1. 
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140 
141 
142 
143 
144 
145 
146 
141 
148 
149 
150 
151 
152 
153 
154 
155 
156 
157 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
171 
172 


JAZ 
LDA 
DECA 
JAZ 
U3 LD6 
LD1 
ENT1 
JMP 
U4A LDA 
JMP 
U4 LDA 
DECA 
JANZ 
LDA 
JANZ 
U6 JMP 
LDA 
STA 
ST6 
JMP 
U5 JMP 
ENT1 
JMP 
ENTA 
LD2 
STA 
J5NZ 
DEC2 
ENTS 
ENT6 
JMP 
ENTA 
JMP 


*+3 


ELEV1+2(NEXTINST) 


E1 
DECISION 
TEMP 
0,6(IN) 
QUEUE, 1 
INSERT 
GIVEUPTIME 
HOLDC 
0,6(IN) 
0,4 

*+3 

D1 

U4A 

DELETE 
AVAIL 
0,6(RLINK1) 
AVAIL 
CYCLE 
DELETE 
ELEVATOR 
INSERT 

1 

3,6 (OUT) 
CALL ,2(3:3) 
CYCLE 

0,4 

0,2 

ELEV2 
DELETEW 

25 

E5A 
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If D2 = 0, call the DECISION subroutine. 


If the elevator is at E1, call 
the DECISION subroutine. 
U3. Enter queue. 


rll + LOC(QUEUE[IN] ). 
Insert NODE(C) at right end of QUEUE[IN]. 


Wait GIVEUPTIME units. 
U4. Give up. 
IN(C) — FLOOR. 


FLOOR = IN(C). 

See exercise 7. 

U6. Get out. NODE(C) is deleted 
from QUEUE or ELEVATOR. 

AVAIL <— C. 


Continue simulation. 

U5. Get in. NODE(C) is deleted 
from QUEUE. 

Insert it at right of ELEVATOR. 


Set CALLCAR[OUT(C)] + 1. 

Jump if STATE x NEUTRAL. 

rl2 + OUT(C) — FLOOR. 

Set STATE to proper direction. 
Set C «+ LOC(ELEV2). 

Remove E5 action from WAIT list. 


Restart E5 action 25 units from now. f 


The program for Coroutine E is a rather straightforward rendition of the 
semiformal description given earlier. Perhaps the most interesting portion is the 
preparation for the elevator’s independent actions in step E3, and the searching 
of the ELEVATOR and QUEUE lists in step E4. 


173 
114 
175 
176 
177 
178 
179 
180 
181 
182 


* COROUTINE E 


E1A JMP 
E1 EQU 
E2A JMP 
E2 J5N 
LDA 
ADD 
ADD 
ADD 
JAP 


CYCLE1 

* 

HOLDC 

1F 
CALL+1,4 
CALL+2,4 
CALL+3,4 
CALL+4,4 
E3 


Set NEXTINST + E1, go to CYCLE. 
E1. Wait for call. (no action) 


E2. Change of state? 
State is GOINGUP. 


Are there calls for higher floors? 
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183 
184 
185 
186 
187 
188 
189 


196 
197 
198 
199 
200 
201 
202 
203 
204 
205 
206 
207 
208 
209 
210 
211 
212 
213 
214 
215 
216 
217 
218 
219 
220 
221 
222 
223 
224 
225 
226 
227 
228 
229 
230 
231 
232 
233 
234 
235 
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1H 


2H 


E3 


E4A 


1H 


2H 


1H 


LDA 
ADD 
ADD 
ADD 
JMP 
LDA 
ADD 


ADD 
ENNS 
STZ 
JANZ 
ENTS 
ENT6 
LDA 
JANZ 
ENTA 
JMP 
ENT6 
ENTA 
JMP 
ST6 
ST6 
ENTA 
ENT6 
JMP 
ENTA 
SLA 
ENT6 
LD6 
CMP6 
JE 
CMPA 
JNE 
ENTA 
JMP 
LD6 
CMP6 
JE 
JMP 
ENTA 
STA 
JMP 
ENTA 
JMP 
STZ 
ST6 
JMP 


CALL-1,4(3:3) 
CALL-2,4(3:3) 
CALL-3 ,4(3:3) 
CALL-4,4(3:3) 
2F 

CALL-1,4 
CALL-2,4 


CALL+4,4(3:3) 
0,5 

CALL, 4 

E3 

0 

ELEV3 

0,6 
DELETEW 
300 

HOLD 

ELEV2 

76 

HOLD 

D2 

D1 

20 

ELEV1 
HOLDC 

0,4 

4 

ELEVATOR 
3,6(LLINK2) 
=ELEVATOR= 
1F 

3,6 (OUT) 

1B 

U6 

2F 
QUEUE+3, 4 (RLINK2) 
3,6(RLINK2) 
1F 

DELETEW 

U5 
2,6(NEXTINST) 
IMMED 

25 

E4A 

D1 

D3 

CYCLE 
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If not, have passengers in the 
elevator called for lower floors? 


State is GOINGDOWN. 
Actions are like lines 178-186. 


Reverse direction of STATE. 

Set CALL variables to zero. 

Jump if called to the opposite direction; 
otherwise set STATE <— NEUTRAL. 

E3. Open doors. 

If activity E9 is already scheduled, 
remove it from the WAIT list. 


Schedule activity E9 after 300 units. 


Schedule activity E5 after 76 units. 
Set D2 nonzero. 
Set D1 nonzero. 


E4. Let people out, in. 

Set OUT field of rA to FLOOR. 

C + LOC (ELEVATOR). 

C + LLINK2(C). 

Search ELEVATOR list, right to left. 

If C = LOC(ELEVATOR), search is complete. 

Compare OUT(C) with FLOOR. 

If not equal, continue searching; 
otherwise prepare to send user to U6. 


Set C + RLINK2(LOC(QUEUE[FLOOR]) ). 
Is C = RLINK2(C)? 

If so, the queue is empty. 

If not, cancel action U4 for this user. 
Prepare to replace U4 by U5. 

Set NEXTINST(C). 

Put user at the front of the WAIT list. 


Wait 25 units and repeat E4. 
Set Di + 0. 

Set D3 nonzero. 

Return to simulate other events. 
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236 E5A JMP 

237 E5 LDA 

238 JAZ 

239 ENTA 
240 JMP 

241 STZ 

242 ENT6 
243 ENTA 
244 JMP 

245 E6 J5N 

246 STZ 

241 J5P 

248 STZ 

249 J5Z 

250 E6B J5Z 

251 LDA 

252 JAZ 

253 ENT6 
254 JMP 

255 STZ 

256 ENT6 
257 ENTA 
258 J5N 

259 ETA JMP 

260 E7 INC4 
261 ENTA 
262 JMP 

263 LDA 

264 JAP 

265 ENT1 
266 J1Z 

267 LDA 

268 JAZ 

269 2H LDA 

270 ADD 

271 ADD 

272 ADD 

273 JANZ 
274 1H ENTA 
275 JMP 

276 E8A JMP 

292 JMP 

293 EQ STZ 

294 STZ 

295 JMP 

296 JMP 


HOLDC 
Di 

*+3 

40 

E5A 

D3 

ELEV1 

20 

HOLDC 

*+2 
CALL,4(1:3) 
*+2 

CALL ,4(3:5) 
DECISION 
E1A 

D2 

*+4 

ELEV3 
DELETEW 
ELEV3 
ELEV1 


51 
HOLDC 
CALL,4(1:3) 
1F 

-2,4 

2F 
CALL,4(5:5) 
E7 
CALL+1,4 
CALL+2,4 
CALL+3,4 
CALL+4,4 
E7 

14 

E2A 

HOLDC 


E2A 

0,6 

D2 
DECISION 
CYCLE 
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E5. Close doors. 

Is D1 = 0? 

If not, people are still getting in or out. 
Wait 40 units, repeat E5. 

If Di = 0, set D3 + 0. 


Wait 20 units, then go to E6. 

E6. Prepare to move. 

If STATE 4 GOINGDOWN, CALLUP and CALLCAR 
on this floor are reset. 

If Æ GOINGUP, reset CALLCAR and CALLDOWN. 

Perform DECISION subroutine. 

If STATE = NEUTRAL, go to El and wait. 


Otherwise, if D2 4 0, 
cancel activity E9 
(see line 202). 


Wait 15 units of time. 
If STATE = GOINGDOWN, go to E8. 


E7. Go up a floor. 


Wait 51 units. 
Is CALLCAR[FLOOR] or CALLUP [FLOOR] x 0? 


If not, 

is FLOOR = 2? 
If not, is CALLDOWN [FLOOR] Æ 0? 
If not, repeat step E7. 


Are there calls for higher floors? 
It is time to stop the elevator. 
Wait 14 units and go to E2. 


(See exercise 8.) 


E9. Set inaction indicator. (See line 202.) 
D2 + 0. 

Perform DECISION subroutine. 

Return to simulation of other events. Į 
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We will not consider here the DECISION subroutine (see exercise 9), nor the 
VALUES subroutine that is used to specify the demands on the elevator. At the 
very end of the program comes the code 


BEGIN ENT4 2 Start with FLOOR = 2 
ENT5 0 and STATE = NEUTRAL. 
JMP CYCLE Begin simulation. 
POOLMAX NOP POOL 
POOL END BEGIN Storage pool follows literals, temp storage. J 


The program above does a fine job of simulating the elevator system, as it 
goes through its paces. But it would be useless to run this program, since there 
is no output! Actually, the author added a PRINT subroutine that was called at 
most of the critical steps in the program above, and this was used to prepare 
Table 1; the details have been omitted, since they are very straightforward but 
they only clutter up the code. 

Several programming languages have been devised that make it quite easy 
to specify the actions in a discrete simulation, and to use a compiler to translate 
these specifications into machine language. Assembly language was used in this 
section, of course, since we are concerned here with the basic techniques of linked 
list manipulation, and we want to see the details of how discrete simulations can 
actually be performed by a computer that has a one-track mind. The technique 
of using a WAIT list or agenda to control the sequencing of coroutines, as we have 
done in this section, is called quasiparallel processing. 

It is quite difficult to give a precise analysis of the running time of such a long 
program, because of the complex interactions involved; but large programs often 
spend most of their time in comparatively short routines doing comparatively 
simple things. Therefore we can usually get a good indication of the overall 
efficiency by using a special trace routine called a profiler, which executes the 
program and records how often each instruction is performed. This identifies the 
“bottlenecks,” the places that should be given special attention. [See exercise 
1.4.3.2-7. See also Software Practice & Experience 1 (1971), 105-133, for 
examples of such studies on randomly selected FORTRAN programs found in 
wastebaskets at the Stanford Computer Center.] The author made such an 
experiment with the elevator program above, running it for 10000 units of 
simulated elevator time; 26 users entered the simulated system. The instructions 
in the SORTIN loop, lines 073-075, were executed by far the most often, 1432 
times, while the SORTIN subroutine itself was called 437 times. The CYCLE 
routine was performed 407 times; so we could gain a little speed by not calling 
the DELETEW subroutine at line 095: The four lines of that subroutine could be 
written out in full (to save 4u each time CYCLE is used). The profiler also showed 
that the DECISION subroutine was called only 32 times and the loop in E4 (lines 
217-219) was executed only 142 times. 

It is hoped that some reader will learn as much about simulation from the 
example above as the author learned about elevators while the example was 
being prepared. 
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EXERCISES 


1. [21] Give specifications for the insertion and deletion of information at the left 
end of a doubly linked list represented as in (1). (With the dual operations at the right 
end, which are obtained by symmetry, we therefore have all the actions of a general 
deque.) 


2. [22] Explain why a list that is singly linked cannot allow efficient operation as a 
general deque; the deletion of items can be done efficiently at only one end of a singly 
linked list. 


3. [22] The elevator system described in the text uses three call variables, CALLUP, 
CALLCAR, and CALLDOWN, for each floor, representing buttons that have been pushed by 
the users in the system. It is conceivable that the elevator actually needs only one or 
two binary variables for the call buttons on each floor, instead of three. Explain how 
an experimenter could push buttons in a certain sequence with this elevator system to 
prove that there are three independent binary variables for each floor (except the top 
and bottom floors). 


4. [24] Activity E9 in the elevator coroutine is usually canceled by step E6; and 
even when it hasn’t been canceled, it doesn’t do very much. Explain under what 
circumstances the elevator would behave differently if activity E9 were deleted from 
the system. Would it, for example, sometimes visit floors in a different order? 


5. [20] In Table 1, user 10 arrived on floor 0 at time 1048. Show that if user 10 had 
arrived on floor 2 instead of floor 0, the elevator would have gone up after receiving 
its passengers on floor 1, instead of down, in spite of the fact that user 8 wants to go 
down to floor 0. 


6. [23] During the time period 1183-1233 in Table 1, users 7, 8, and 9 all get in the 
elevator on floor 1; then the elevator goes down to floor 0 and only user 8 gets out. 
Now the elevator stops again on floor 1, presumably to pick up users 7 and 9 who are 
already aboard; nobody is actually on floor 1 waiting to get in. (This situation occurs 
not infrequently at Caltech; if you get on the elevator going the wrong way, you must 
wait for an extra stop as you go by your original floor again.) In many elevator systems, 
users 7 and 9 would not have boarded the elevator at time 1183, since lights outside 
the elevator would show that it was going down, not up; those users would have waited 
until the elevator came back up and stopped for them. On the system described, there 
are no such lights and it is impossible to tell which way the elevator is going to go until 
you are in it; hence Table 1 reflects the actual situation. 

What changes should be made to coroutines U and E if we were to simulate the 
same elevator system, but with indicator lights, so that people do not get on the elevator 
when its state is contrary to their desired direction? 


7. [25] Although bugs in programs are often embarrassing to a programmer, if we 
are to learn from our mistakes we should record them and tell other people about them 
instead of forgetting them. The following error (among others) was made by the author 
when he first wrote the program in this section: Line 154 said ‘JANZ CYCLE’ instead 
of ‘JANZ U4A’. The reasoning was that if indeed the elevator had arrived at this user’s 
floor, there was no need to perform the “give up” activity U4 any more, so we could 
simply go to CYCLE and continue simulating other activities. What was the error? 


8. [21] Write the code for step E8, lines 277-292, which has been omitted from the 
program in the text. 
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9. [23] Write the code for the DECISION subroutine, which has been omitted from 
the program in the text. 


10. [40] It is perhaps significant to note that although the author had used the 
elevator system for years and thought he knew it well, it wasn’t until he attempted 
to write this section that he realized there were quite a few facts about the elevator’s 
system of choosing directions that he did not know. He went back to experiment with 
the elevator six separate times, each time believing he had finally achieved a complete 
understanding of its modus operandi. (Now he is reluctant to ride it for fear that some 
new facet of its operation will appear, contradicting the algorithms given.) We often 
fail to realize how little we know about a thing until we attempt to simulate it on a 
computer. 

Try to specify the actions of some elevator you are familiar with. Check the 
algorithm by experiments with the elevator itself (looking at its circuitry is not fair!); 
then design a discrete simulator for the system and run it on a computer. 


11. [21] (A sparse-update memory.) The following problem often arises in synchro- 
nous simulations: The system has n variables V[1], ..., Vin], and at every simulated 
step new values for some of them are calculated from the old values. These calculations 
are assumed done “simultaneously” in the sense that the variables do not change to 
their new values until after all assignments have been made. Thus, the two statements 


v[1] + V[2] and v[2] + vV[1] 


appearing at the same simulated time would interchange the values of V[1] and V[2]; 
this is quite different from what would happen in a sequential calculation. 

The desired action can of course be simulated by keeping an additional table 
NEW [1], ..., NEW [n]. Before each simulated step, we could set NEWV[k] <+ V[k] for 
1 < k < n, then record all changes of V[k] in NEW [k], and finally, after the step 
we could set V[k] 4+ NEW [k], 1 < k < n. But this “brute force” approach is not 
completely satisfactory, for the following reasons: (1) Often n is very large, but the 
number of variables changed per step is rather small. (2) The variables are often not 
arranged in a nice table V[1],...,V[n], but are scattered throughout memory in a 
rather chaotic fashion. (3) This method does not detect the situation (usually an error 
in the model) when one variable is given two values in the same simulated step. 

Assuming that the number of variables changed per step is rather small, design 
an efficient algorithm that simulates the desired actions, using two auxiliary tables 
NEWV [k] and LINK[k], 1 < k < n. If possible, your algorithm should give an error stop 
if the same variable is being given two different values in the same step. 


12. [22] Why is it a good idea to use doubly linked lists instead of singly linked or 
sequential lists in the simulation program of this section? 


2.2.6. Arrays and Orthogonal Lists 


One of the simplest generalizations of a linear list is a two-dimensional or higher- 
dimensional array of information. For example, consider the case of an m x n 
matrix 


A[C1,1] AC,2] ... ACl,n] 


A[2,1] A[2,2] ... A[2,n] 
i : . (1) 


Alm,1] Alm,2].... Alm,n] 
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In this two-dimensional array, each node A[j,k] belongs to two linear lists: 
the “row j” list ALj,1], ALj,2],..., Alj,n] and the “column k” list A[1,4], 
A(2,k],...,Alm,k]. These orthogonal row and column lists essentially account 
for the two-dimensional structure of a matrix. Similar remarks apply to higher- 
dimensional arrays of information. 


Sequential Allocation. When an array like (1) is stored in sequential memory 
locations, storage is usually allocated so that 


LOC (ALJ, K]) = ao + aıJ + a2K, (2) 


where ao, a1, and az are constants. Let us consider a more general case: 
Suppose we have a four-dimensional array with one-word elements Q[I,J,K,L] 
fr0<I<2,0<J<40<KK<10,0<LK<2. We would like to allocate 
storage so that 


LOC(Q[I,J,K,L]) = ao aI } agJ t a3k t a4L. (3) 


This means that a change in I, J, K, or L leads to a readily calculated change in 
the location of Q(I,J,K,L]. The most natural (and most commonly used) way 
to allocate storage is to arrange the array elements according to the lexicographic 
order of their indices (exercise 1.2.1-15(d)), sometimes called “row major order”: 


Q(0,0,0,0], Q[0,0,0,1], Q[0,0,0,2], Q[0,0,1,0], Q[0,0,1,1], yery 


Q[0,0,10,2], Q[0,1,0,0], ..., Q[0,4,10,2], Q[1,0,0,0], ..., 
Q[2,4,10,2]. 
It is easy to see that this order satisfies the requirements of (3), and we have 
LOC(Q[I,J,K,L]) = LOC(Q(0,0,0,0]) + 165I + 33J + 3K +L. (4) 
In general, given a k-dimensional array with c-word elements A[I,,I2,...,Ix] 
for 
0O<I <d, 0<I2<d2, ..., 0< Ik < dk, 
we can store it in memory as 
LOC(A[I,,Io,...,14]) 
=LOC(A[0,0,...,0])+c(d2+1)...(dk+1)I1+:--+c(dk+1)Ik-1 +cIk 
=L0C(A[0,0,...,0]1)+ > aIr, (5) 
1<r<k 
where 
ay =c [|| (+0). (6) 


r<s<k 


To see why this formula works, observe that a, is the amount of memory needed 


to store the subarray A[I,,...,1,,Jp41,.--,Jz] if 1,,...,1, are constant and 
Jr+1;---, Jk vary through all values 0 < Jp4y < dpi, -.-, 0 < Jk < dk; hence 
by the nature of lexicographic order the address of A[I;,...,1,] should change 


by precisely this amount when I, changes by 1. 
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Formulas (5) and (6) correspond to the value of the number I,Ig...I, ina 
mixed-radix number system. For example, if we had the array TIME[W,D,H,M,S] 
withO<W<4,0<D<7,0<H < 24,0 <™M < 60, and 0 <S < 60, the 
location of TIME[W,D,H,M,S] would be the location of TIME[0,0,0,0,0] plus 
the quantity “W weeks +D days +H hours +M minutes + S seconds” converted to 
seconds. Of course, it takes a pretty fancy application to make use of an array 
that has 2,419,200 elements. 

The normal method for storing arrays is generally suitable when the array 
has a complete rectangular structure, so that all elements A[I,,I2,...,I,] are 
present for indices in the independent ranges lı < Iq < u1, l2 < I2 < ue, ..., 
I, < Ip < ux. Exercise 2 shows how to adapt (5) and (6) to the case when the 
lower bounds (l1, l2,...,1,) are not (0,0,...,0). 

But there are many situations in which an array is not perfectly rectangular. 
Most common is the triangular matrix, where we want to store only the entries 
ALj,k] for, say,O<k<j<n: 


A[0,0] 


AtM1,0] A(i,1) 
(7) 


A[n,0] Aln,1]... Aln,n] 


We may know that all other entries are zero, or that A[j,k] = A[k,j], so that 
only half of the values need to be stored. If we want to store the lower triangular 
matrix (7) in $(n + 1)(n + 2) consecutive memory positions, we are forced to 
give up the possibility of linear allocation as in Eq. (2), but we can ask instead 
for an allocation arrangement of the form 


LOC(A[J,K]) = ao + fı (J) + foCK) (8) 


where fı and f2 are functions of one variable. (The constant aj may be absorbed 
into either fı or fz if desired.) When the addressing has the form (8), a random 
element A[j,k] can be quickly accessed if we keep two (rather short) auxiliary 
tables of the values of fı and f2; therefore these functions need to be calculated 
only once. 

It turns out that lexicographic order of indices for the array (7) satisfies 
condition (8), and with one-word entries we have in fact the simple formula 


J(J +1) 


LOC(A[J,K]) = LOC(A[O,0]) + 5 


+K. (9) 

But there is actually a far better way to store triangular matrices, if we are 
fortunate enough to have two of them with the same size. Suppose that we want 
to store both A[j,k] and B[j,k] forO <k <j <n. Then we can fit them both 
into a single matrix C[j,k] for 0< j < n, 0 <k <n +1, using the convention 


ALj, kl =CLj,k],  BIj,k] =C[k,j+1]. (10) 
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Thus 
c[0,0] C[0,1] C[0,2] ...cl0,n+1] A[0,0] B[0,0] B[1,0] ...B[n,0] 
C[1,0] C[1,1] C[1,2] ...C[L,n+1] | j| 40,0] A[,1] BO,1)...Bln,] 
Cin, 0] Cin, 1] Cin, 2] ...Cin,n+1] Aln, 0] Aln, 1] Aln, 2] ...BIn,n] 


The two triangular matrices are packed together tightly within the space of 
(n + 1)(n + 2) locations, and we have linear addressing as in (2). 

The generalization of triangular matrices to higher dimensions is called a 
tetrahedral array. This interesting topic is the subject of exercises 6 through 8. 

As an example of typical programming techniques for use with sequentially 
stored arrays, see exercise 1.3.2-10 and the two answers given for that exercise. 
The fundamental techniques for efficient traversal of rows and columns, as well 
as the uses of sequential stacks, are of particular interest within those programs. 


Linked Allocation. Linked memory allocation also applies to higher-dimen- 
sional arrays of information in a natural way. In general, our nodes can contain 
k link fields, one for each list the node belongs to. The use of linked memory is 
generally for cases in which the arrays are not strictly rectangular in character. 


[$à] 
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PERSON [6] 


Female, age 21, brown eyes, dark hair 


PERSON [5] 


Male, age 24, brown eyes, dark hair 


PERSON [4] 


Female, age 22, green eyes, blonde hair 


PERSON [3] 


Male, age 28, hazel eyes, blond hair 


PERSON [2] 


Female, age 22, blue eyes, red hair 


PERSON [1] 


Female, age 21, blue eyes, blonde hair 


Fig. 13. Each node in four different lists. 


As an example, we might have a list in which every node represents a person, 
with four link fields: SEX, AGE, EYES, and HAIR. In the EYES field we link together 
all nodes with the same eye color, etc. (See Fig. 13.) It is easy to visualize efficient 
algorithms for inserting new people into the list; deletion would, however, be 
much slower, unless we used double linking. We can also conceive of algorithms 
of varying degrees of efficiency for doing things like “Find all blue-eyed blonde 
women of ages 21 through 23”; see exercises 9 and 10. Problems in which each 
node of a list is to reside in several kinds of other lists at once arise rather 
frequently; indeed, the elevator system simulation described in the preceding 
section has nodes that are in both the QUEUE and WAIT lists simultaneously. 
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As a detailed example of the use of linked allocation for orthogonal lists, 
we will consider the case of sparse matrices (that is, matrices of large order in 
which most of the elements are zero). The goal is to operate on these matrices 
as though the entire matrix were present, but to save great amounts of time 
and space because the zero entries need not be represented. One way to do 
this, intended for random references to elements of the matrix, would be to use 
the storage and retrieval methods of Chapter 6, to find A[j,k] from the key 
“14, k]”; however, there is another way to deal with sparse matrices that is often 
preferable because it reflects the matrix structure more appropriately, and this 
is the method we will discuss here. 

The representation we will discuss consists of circularly linked lists for each 
row and column. Every node of the matrix contains three words and five fields: 


ROW UP 


COL LEFT (11) 


VAL 


Here ROW and COL are the row and column indices of the node; VAL is the value 
stored at that part of the matrix; LEFT and UP are links to the next nonzero 
entry to the left in the row, or upward in the column, respectively. There are 
special list head nodes, BASEROW [i] and BASECOL[ j], for every row and column. 
These nodes are identified by 


COL (LOC (BASEROW[i])) < 0 and ROW (LOC(BASECOL[j])) < 0. 


As usual in a circular list, the LEFT link in BASEROW[i¿] is the location of the 
rightmost value in that row, and UP in BASECOL[7] points to the bottom-most 
value in that column. For example, the matrix 


50 0 00 
10 0 20 0 

00 00 (12) 
-30 0 —60 5 


would be represented as shown in Fig. 14. 

Using sequential allocation of storage, a 200 x 200 matrix would take 40000 
words, and this is more memory than many computers used to have; but a 
suitably sparse 200 x 200 matrix can be represented as above even in MIX’s 
4000-word memory. (See exercise 11.) The amount of time taken to access a 
random element A[j,k] is also quite reasonable, if there are but few elements 
in each row or column; and since most matrix algorithms proceed by walking 
sequentially through a matrix, instead of accessing elements at random, this 
linked representation often works faster than a sequential one. 

As a typical example of a nontrivial algorithm dealing with sparse matrices 
in this form, we will consider the pivot step operation, which is an important part 
of algorithms for solving linear equations, for inverting matrices, and for solving 
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Fig. 14. Representation of matrix (12), with nodes in the format 
List heads appear at the left and at the top. 


ROW | COL | VAL 


linear programming problems by the simplex method. A pivot step is the follow- 
ing matrix transformation (see M. H. Doolittle, Report of the Superintendent of 
the U. S. Coast and Geodetic Survey (1878), 115-120): 


Before pivot step After pivot step 
Any Any 
Pivot other Pivot other 
column column column column 
Pivot row Seca a sei b er oe 1/a soa b/a 
Any other row] --- c ++: d oe s+ =c/a +++ d—be/a 


(13) 


It is assumed that the pivot element, a, is nonzero. For example, a pivot step 
applied to matrix (12), with the element 10 in row 2 column 1 as pivot, leads to 


-5 0 —100 0 
01 0 2 0 
0 0 0 0 (14) 
3 0 05 
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Our goal is to design an algorithm that performs this pivot operation on 
sparse matrices that are represented as in Fig. 14. It is clear that the trans- 
formation (13) affects only those rows of a matrix for which there is a nonzero 
element in the pivot column, and it affects only those columns for which there 
is a nonzero entry in the pivot row. 

The pivoting algorithm is in many ways a straightforward application of 
linking techniques we have already discussed; in particular, it bears strong 
resemblances to Algorithm 2.2.4A for addition of polynomials. There are two 
things, however, that make the problem a little tricky: If in (13) we have b 4 0 
and c Æ 0 but d = 0, the sparse matrix representation has no entry for d and we 
must insert a new entry; and if b #0, c 40, d #0, but d — bc/a = 0, we must 
delete the entry that was formerly there. These insertion and deletion operations 
are more interesting in a two-dimensional array than in the one-dimensional case; 
to do them we must know what links are affected. Our algorithm processes the 
matrix rows successively from bottom to top. The efficient ability to insert and 
delete involves the introduction of a set of pointer variables PTR[j], one for each 
column considered; these variables traverse the columns upwards, giving us the 
ability to update the proper links in both dimensions. 


Algorithm S (Pivot step in a sparse matriz). Given a matrix represented as 
in Fig. 14, we perform the pivot operation (13). Assume that PIVOT is a link 
variable pointing to the pivot element. The algorithm makes use of an auxiliary 
table of link variables PTR[j], one for each column of the matrix. The variable 
ALPHA and the VAL field of each node are assumed to be floating point or rational 
quantities, while everything else in this algorithm has integer values. 


S1. [Initialize.] Set ALPHA + 1.0/VAL(PIVOT), VAL(PIVOT) + 1.0, and 


IO + ROW(PIVOT), PO < LOC(BASEROW[IO]); 
JO + COL(PIVOT), QO + LOC(BASECOL[JO]). 


S2. [Process pivot row.] Set PO «+ LEFT(PO), J + COL(PO). If J < 0, go on 
to step $3 (the pivot row has been traversed). Otherwise set PTR[J] + 
LOC(BASECOL[J]) and VAL(PO) «+ ALPHA x VAL(PO), and repeat step S2. 


S3. [Find new row.] Set QO + UP(QO). (The remainder of the algorithm deals 
successively with each row, from bottom to top, for which there is an entry 
in the pivot column.) Set I + ROW(QO). If I < 0, the algorithm terminates. 
If I = 10, repeat step S3 (we have already done the pivot row). Otherwise 
set P + LOC(BASEROW[I]), P1 + LEFT(P). (The pointers P and P1 will now 
proceed across row I from right to left, as PO goes in synchronization across 
row 10; Algorithm 2.2.4A is analogous. We have PO = LOC(BASEROW[I0]) 
at this point.) 


S4. [Find new column.] Set PO + LEFT(PO), J «+ COL(PO). If J < 0, set 
VAL(QO) + —ALPHA x VAL(QO) and return to 83. If J = JO, repeat step S4. 
(Thus we process the pivot column entry in row I after all other column 
entries have been processed; the reason is that VAL (QO) is needed in step S7.) 
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S5. [Find I, J element.] If COL(P1) > J, set P+ P1, P1 + LEFT(P), and repeat 
step S5. If COL(P1) = J, go to step S7. Otherwise go to step S6 (we need 
to insert a new element in column J of row I). 


S6. [Insert I, J element.] If ROW(UP(PTR[J])) > I, set PTR[J] + UP(PTRI[J]), 
and repeat step S6. (Otherwise, we will have ROW(UP(PTR[J])) < I; the new 
element is to be inserted just above NODE(PTR[J] ) in the vertical dimension, 
and just left of NODE(P) in the horizontal dimension.) Otherwise set X <= 
AVAIL, VAL(X) + 0, ROW(X) + I, COL(X) © J, LEFT(X) + P1, UP(X) © 
UP(PTR[J]), LEFT(P) + X, UP(PTR[J]) + X, P1 + X. 

S7. [Pivot.] Set VAL(P1) + VAL (P1) —VAL(QO) x VAL(PO). If now VAL(P1) = 0, 
go to S8. (Note: When floating point arithmetic is being used, this test 
“VAL(P1) = 0” should be replaced by “|VAL(P1)| < EPSILON” or better yet 
by the condition “most of the significant figures of VAL(P1) were lost in the 
subtraction.”) Otherwise, set PTR[J] + P1, P + P1, Pi + LEFT(P), and go 
back to S4. 


S8. [Delete I, J element.] If UP(PTR[J]) # P1 (or, what is essentially the same 
thing, if ROWCUP(PTR[J])) > I), set PTR[J] + UP(PTR[J]) and repeat 
step S8; otherwise, set UP(PTR[J]) < UP(P1), LEFT(P) < LEFT(P1), 
AVAIL = P1, P1 + LEFT(P). Go back to S4. J 


The programming of this algorithm is left as a very instructive exercise for 
the reader (see exercise 15). It is worth pointing out here that it is necessary to 
allocate only one word of memory to each of the nodes BASEROW Li] , BASECOL[7], 
since most of their fields are irrelevant. (See the shaded areas in Fig. 14, and see 
the program of Section 2.2.5.) Furthermore, the value —PTR[j] can be stored as 
ROW (LOC (BASECOL[j])) for additional storage space economy. The running time 
of Algorithm S is very roughly proportional to the number of matrix elements 
affected by the pivot operation. 

This representation of sparse matrices via orthogonal circular lists is in- 
structive, but numerical analysts have developed better methods. See Fred G. 
Gustavson, ACM Trans. on Math. Software 4 (1978), 250-269; see also the graph 
and network algorithms in Chapter 7 (for example, Algorithm 7B). 


EXERCISES 


1. [17] Give a formula for LOC(A[J,K]) if A is the matrix of (1), and if each node 
of the array is two words long, assuming that the nodes are stored consecutively in 
lexicographic order of the indices. 

2. [21] Formulas (5) and (6) have been derived from the assumption that 0 < I, < dr 
for 1 < r < k. Give a general formula that applies to the case lp < I, < ur, where 
l, and ur are any lower and upper bounds on the dimensionality. 

3. [21] The text considers lower triangular matrices A[j,k] forO < k <j <n. How 
can the discussion of such matrices readily be modified for the case that subscripts 
start at 1 instead of 0, so that 1 < k < j < n? 

4. [22] Show that if we store the upper triangular array A[j,k] forO<j<k<n 
in lexicographic order of the indices, the allocation satisfies the condition of Eq. (8). 
Find a formula for LOC(A[LJ,K]) in this sense. 
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5. [20] Show that it is possible to bring the value of A[J,K] into register A in one 
MIX instruction, using the indirect addressing feature of exercise 2.2.2-3, even when 
A is a triangular matrix as in (g). (Assume that the values of J and K are in index 
registers. ) 

6. [M24] Consider the “tetrahedral arrays” A[i, j,k], Bli, j,k], where 0 < k < j < 
i < nin A, and0 <i< j <k < ninB. Suppose that both of these arrays are 
stored in consecutive memory locations in lexicographic order of the indices; show that 
LOC(A[I,J,K]) = ao + fı(I) + f2(J) + fs(K) for certain functions fi, fo, fg. Can 
LOC(B[I,J,K]) be expressed in a similar manner? 


7. [M23] Find a general formula to allocate storage for the k-dimensional tetrahedral 
array A[ii,272,...,th], where 0 < ik <- Liliu <n. 

8. [33] (P. Wegner.) Suppose we have six tetrahedral arrays A[I,J,K], B[I,J,K], 
C{I,J,K], D[I,J,K], E[I,J,K], and F[I,J,K] to store in memory, where 0<K<J< 
I <n. Is there a neat way to accomplish this, analogous to (10) in the two-dimensional 
case? 


9. [22] Suppose a table, like that indicated in Fig. 13 but much larger, has been 
set up so that all links go in the same direction as shown there (namely, LINK(X) < X 
for all nodes and links). Design an algorithm that finds the addresses of all blue-eyed 
blonde women of ages 21 through 23, by going through the various link fields in such 
a way that upon completion of the algorithm at most one pass has been made through 
each of the lists FEMALE, A21, A22, A23, BLOND, and BLUE. 


10. [26] Can you think of a better way to organize a personnel table so that searches 
as described in the previous exercise would be more efficient? (The answer to this 
exercise is not merely “yes” or “no.”) 

11. [11] Suppose that we have a 200 x 200 matrix in which there are at most four 
nonzero entries per row. How much storage is required to represent this matrix as in 
Fig. 14, if we use three words per node except for list heads, which will use one word? 
12. [20] What are VAL(QO), VAL(PO), and VAL(P1) at the beginning of step S7, in 
terms of the notation a, b, c, d used in (13)? 


13. [22] Why were circular lists used in Fig. 14 instead of straight linear lists? Could 
Algorithm S be rewritten so that it does not make use of the circular linkage? 


14. [22] Algorithm S actually saves pivoting time in a sparse matrix, since it avoids 
consideration of those columns in which the pivot row has a zero entry. Show that 
this savings in running time can be achieved in a large sparse matrix that is stored 
sequentially, with the help of an auxiliary table LINK[j], 1 < j <n. 

15. [29] Write a MIXAL program for Algorithm S. Assume that the VAL field is a 
floating point number, and that MIX’s floating point arithmetic operators FADD, FSUB, 
FMUL, and FDIV can be used for operations on this field. Assume for simplicity that 
FADD and FSUB return the answer zero when the operands added or subtracted cancel 
most of the significance, so that the test “VAL(P1) = 0” may safely be used in step S7. 
The floating point operations use only rA, not rX. 


16. [25] Design an algorithm to copy a sparse matrix. (In other words, the algorithm 
is to yield two distinct representations of a matrix in memory, having the form of 
Fig. 14, given just one such representation initially.) 

17. [26] Design an algorithm to multiply two sparse matrices; given matrices A and B, 
form a new matrix C, where C[i,7] = a Ali, k]B(k,7]. The two input matrices and 
the output matrix should be represented as in Fig. 14. 
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18. [22] The following algorithm replaces a matrix by the inverse of that matrix, 
assuming that the entries are A[t,j], for 1 < i,j < n: 


i) For k = 1, 2, ..., n do the following: Search row k in all columns not yet used 
as a pivot column, to find an entry with the greatest absolute value; set C[k] equal to 
the column in which this entry was found, and do a pivot step with this entry as pivot. 
(If all such entries are zero, the matrix is singular and has no inverse.) 

ii) Permute rows and columns so that what was row k becomes row C[k], and 
what was column C[k] becomes column k. 


The problem in this exercise is to use the stated algorithm to invert the matrix 


1 2 3 
0 1 2 
0 0 1 


by hand calculation. 


19. [31] Modify the algorithm described in exercise 18 so that it obtains the inverse 
of a sparse matrix that is represented in the form of Fig. 14. Pay special attention to 
making the row- and column-permutation operations of step (ii) efficient. 


20. [20] A tridiagonal matrix has entries aj; that are zero except when |i — j| < 1, 
for 1 < i,j < n. Show that there is an allocation function of the form 


LOC(A[I,J]) = ao +aiI+a2J, |I-J| <1, 


which represents all of the relevant elements of a tridiagonal matrix in (3n — 2) consec- 
utive locations. 


21. [20] Suggest a storage allocation function for n x n matrices where n is variable. 
The elements A[I,J] for 1 < I,J < n should occupy n? consecutive locations, regardless 
of the value of n. 


22. [M25] (P. Chowla, 1961.) Find a polynomial p(i1,...,i%) that assumes each 
nonnegative integer value exactly once as the indices (i1,..., ik) run through all k- 
dimensional nonnegative integer vectors, with the additional property that 71+---+i, < 
ji +--+ + jk implies p(i1,..., ik) < p(j1,---,Je)- 

23. [23] An extendible matrix is initially 1 x 1, then it grows from size m x n either 
to size (m +1) x n or to size m x (n + 1) by adding either a new row or a new column. 
Show that such a matrix can be given a simple allocation function in which the elements 
A[I,J] occupy mn consecutive locations, for 0 < I < m and 0 < J < n; no elements 
change location when the matrix grows. 


24. [25] (The sparse array trick.) Suppose you want to use a large array for random 
access, although you won’t actually be referring to very many of its entries. You want 
A[k] to be zero the first time you access it, yet you don’t want to spend the time to 
set every location to zero. Explain how it is possible to read and write any desired 
elements A[k] reliably, given k, without assuming anything about the actual initial 
memory contents, by doing only a small fixed number of additional operations per 
array access. 
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2.3. TREES 


WE NOW TURN to a study of trees, the most important nonlinear structures 
that arise in computer algorithms. Generally speaking, tree structure means a 
“branching” relationship between nodes, much like that found in the trees of 
nature. 

Let us define a tree formally as a finite set T of one or more nodes such that 


a) there is one specially designated node called the root of the tree, root(T); 


and 
b) the remaining nodes (excluding the root) are partitioned into m > 0 disjoint 
sets T,,..., Tm, and each of these sets in turn is a tree. The trees T4, ..., Tm 


are called the subtrees of the root. 


The definition just given is recursive: We have defined a tree in terms of 
trees. Of course, there is no problem of circularity involved here, since trees 
with one node must consist of only the root, and trees with n > 1 nodes are 
defined in terms of trees with fewer than n nodes; hence the concept of a tree 
with two nodes, three nodes, or ultimately any number of nodes, is determined 
by the definition given. There are nonrecursive ways to define trees (for example, 
see exercises 10, 12, and 14, and Section 2.3.4), but a recursive definition seems 
most appropriate since recursion is an innate characteristic of tree structures. 
The recursive character of trees is present also in nature, since buds on young 
trees eventually grow into subtrees with buds of their own, and so on. Exercise 3 
illustrates how to give rigorous proofs of important facts about trees based on a 
recursive definition such as the one above, by using induction on the number of 
nodes in a tree. 

It follows from our definition that every node of a tree is the root of some 
subtree contained in the whole tree. The number of subtrees of a node is called 
the degree of that node. A node of degree zero is called a terminal node, or 
sometimes a leaf. A nonterminal node is often called a branch node. The level 
of a node with respect to T is defined recursively: The level of root(T) is zero, 
and the level of any other node is one higher than that node’s level with respect 
to the subtree of root(T) containing it. 

These concepts are illustrated in Fig. 15, which shows a tree with seven 
nodes. The root is A, and it has the two subtrees {B} and {C, D, E, F,G}. The 
tree {C, D, E, F,G} has node C as its root. Node C is on level 1 with respect to 
the whole tree, and it has three subtrees {D}, {E}, and {F, G}; therefore C has 
degree 3. The terminal nodes in Fig. 15 are B, D, E, and G; F is the only node 
with degree 1; G is the only node with level 3. 

If the relative order of the subtrees T;,...,Tm in (b) of the definition is 
important, we say that the tree is an ordered tree; when m > 2 in an ordered 
tree, it makes sense to call To the “second subtree” of the root, etc. Ordered trees 
are also called “plane trees” by some authors, since the manner of embedding 
the tree in a plane is relevant. If we do not care to regard two trees as different 
when they differ only in the respective ordering of subtrees of nodes, the tree 
is said to be oriented, since only the relative orientation of the nodes, not their 


2.3 TREES 309 


Level 3 © 
Level 2 Q AEF 
Level1 (B) © 

Level 0 (A) 


Fig. 15. A tree. Fig. 16. Another tree. 


order, is being considered. The very nature of computer representation defines 
an implicit ordering for any tree, so in most cases ordered trees are of greatest 
interest to us. We will therefore tacitly assume that all trees we discuss are 
ordered, unless explicitly stated otherwise. Accordingly, the trees of Figs. 15 
and 16 will generally be considered to be different, although they would be the 
same as oriented trees. 

A forest is a set (usually an ordered set) of zero or more disjoint trees. 
Another way to phrase part (b) of the definition of tree would be to say that the 
nodes of a tree excluding the root form a forest. 

There is very little distinction between abstract forests and trees. If we 
delete the root of a tree, we have a forest; conversely, if we add just one node to 
any forest and regard the trees of the forest as subtrees of the new node, we get a 
tree. Therefore the words tree and forest are often used almost interchangeably 
during informal discussions about data structures. 


(b) 


Fig. 17. How shall we draw a tree? 


Trees can be drawn in many ways. Besides the diagram of Fig. 15, three of 
the principal alternatives are shown in Fig. 17, depending on where the root is 
placed. It is not a frivolous joke to worry about how tree structures are drawn 
in diagrams, since there are many occasions in which we want to say that one 
node is “above” or “higher than” another node, or to refer to the “rightmost” 
element, etc. Certain algorithms for dealing with tree structures have become 
known as “top down” methods, as opposed to “bottom up.” Such terminology 
leads to confusion unless we adhere to a uniform convention for drawing trees. 

It may seem that the form of Fig. 15 would be preferable simply because 
that is how trees grow in nature; in the absence of any compelling reason to 
adopt any of the other three forms, we might as well adopt nature’s time-honored 
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Fig. 18. Family trees: (a) pedigree; 
(b) lineal chart. [References: Burke’s 
Peerage (1959); Almanach de Gotha 
(1871); Genealogisches Handbuch des 
Adels: Fiirstliche Hauser, 1; Genesis 
10:1-25.] 
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tradition. With real trees in mind, the author consistently followed a root-at-the- 
bottom convention as the present set of books was first being prepared, but after 
two years of trial it was found to be a mistake: Observations of the computer 
literature and numerous informal discussions with computer scientists about a 
wide variety of algorithms showed that trees were drawn with the root at the 
top in more than 80 percent of the cases examined. There is an overwhelming 
tendency to make hand-drawn charts grow downwards instead of upwards (and 
this is easy to understand in view of the way we write); even the word “subtree,” 
as opposed to “supertree,” tends to connote a downward relationship. From 
these considerations we conclude that Fig. 15 is upside down. Henceforth we 
will almost always draw trees as in Fig. 17(b), with the root at the top and 
leaves at the bottom. Corresponding to this orientation, we should perhaps call 
the root node the apex of the tree, and speak of nodes at shallow and deep levels. 

It is necessary to have good descriptive terminology for talking about trees. 
Instead of making somewhat ambiguous references to “above” and “below,” we 
generally use genealogical words taken from the terminology of family trees. 
Figure 18 shows two common types of family trees. The two types are quite 
different: A pedigree shows the ancestors of a given individual, while a lineal 
chart shows the descendants. 

If “cross-breeding” occurs, a pedigree is not really a tree, because different 
branches of a tree (as we have defined it) can never be joined together. To 
compensate for this discrepancy, Fig. 18(a) mentions Queen Victoria and Prince 
Albert twice in the sixth generation; King Christian IX and Queen Louise 
actually appear in both the fifth and sixth generations. A pedigree can be 
regarded as a true tree if each of its nodes represents “a person in the role of 
mother or father of so-and-so,” not simply a person as an individual. 

Standard terminology for tree structures is taken from the second form of 
family tree, the lineal chart: Each root is said to be the parent of the roots of 
its subtrees, and the latter are said to be siblings; they are children of their 
parent. The root of the entire tree has no parent. For example, in Fig. 19, 
C has three children, D, E, and F; E is the parent of G; B and C are siblings. 
Extension of this terminology — for example, A is the great-grandparent of G; 
B is an aunt or uncle of F; H and F are first 
cousins — is clearly possible. Some authors use the 
masculine designations “father, son, brother” in- (A) 
stead of “parent, child, sibling”; others use “mother, (B) C 
daughter, sister.” In any case a node has at most 
one parent or progenitor. We use the words ancestor G) D (D) 2 ©) 
and descendant to denote a relationship that may 
span several levels of the tree: The descendants of C 
in Fig. 19 are D, E, F, and G; the ancestors of G are 
E, C, and A. Sometimes, especially when talking 
about “nearest common ancestors,” we consider a node to be an ancestor of itself 
(and a descendant of itself); the inclusive ancestors of G are G, E, C, and A, 
while its proper ancestors are just E, C, and A. 


Fig. 19. Conventional 
tree diagram. 
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The pedigree in Figure 18(a) is an example of a binary tree, which is another 
important type of tree structure. The reader has undoubtedly seen binary trees 
in connection with tennis tournaments or other sporting events. In a binary tree 
each node has at most two subtrees; and when only one subtree is present, we 
distinguish between the left and right subtree. More formally, let us define a 
binary tree as a finite set of nodes that either is empty, or consists of a root 
and the elements of two disjoint binary trees called the left and right subtrees 
of the root. 

This recursive definition of binary tree should be studied carefully. Notice 
that a binary tree is not a special case of a tree; it is another concept entirely 
(although we will see many relations between the two concepts). For example, 
the binary trees 

(A) @ 


and 1 

D D (1) 

are distinct —the root has an empty right subtree in one case and a nonempty 

right subtree in the other—although as trees these diagrams would represent 

identical structures. A binary tree can be empty; a tree cannot. Therefore we 

will always be careful to use the word “binary” to distinguish between binary 

trees and ordinary trees. Some authors define binary trees in a slightly different 
manner (see exercise 20). 

Tree structure can be represented graphically in several other ways bearing 
no resemblance to actual trees. Figure 20 shows three diagrams that reflect the 
structure of Fig. 19: Figure 20(a) essentially represents Fig. 19 as an oriented 
tree; this diagram is a special case of the general idea of nested sets, namely 
a collection of sets in which any pair of sets is either disjoint or one contains 
the other. (See exercise 10.) Part (b) of the figure shows nested sets in a line, 
much as part (a) shows them in a plane; in part (b) the ordering of the tree is 
also indicated. Part (b) may also be regarded as an outline of an algebraic 
formula involving nested parentheses. Part (c) shows still another common 
way to represent tree structure, using indentation. The number of different 
representation methods in itself is ample evidence for the importance of tree 
structures in everyday life as well as in computer programming. Any hierarchical 
classification scheme leads to a tree structure. 


å (A(B(A)(J))(C(D)(E(G@))(F))) 


(a) (b) (c) 
Fig. 20. Further ways to show tree structure: (a) nested sets; (b) nested parentheses; 
(c) indentation. 
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An algebraic formula defines an implicit tree structure that is often conveyed 
by other means instead of, or in addition to, the use of parentheses. For example, 
Figure 21 shows a tree corresponding to the arithmetic expression 


a—W(c/d+e/f). (2) 


Standard mathematical conventions, according to which multiplication and divi- 
sion take precedence over addition and subtraction, allow us to use a simplified 
form like (2) instead of the fully parenthesized form “a — (b x ((c/d) + (e/f)))”. 
This connection between formulas and trees is very important in applications. 


Fig. 21. Tree representation of formula (2). 


Notice that the indented list in Fig. 20(c) looks very much like the table 
of contents in a book. Indeed, this book itself has a tree structure; the tree 
structure of Chapter 2 is shown in Fig. 22. Here we notice a significant idea: 
The method used to number sections in this book is another way to specify tree 
structure. Such a method is often called “Dewey decimal notation” for trees, by 
analogy with the similar classification scheme of this name used in libraries. The 
Dewey decimal notation for the tree of Fig. 19 is 


1A; 11B; 111H; 1127; 1.20; 
1.2.1 D; 1.2.2 E; 1.2.2.1 G; 1.2.3 F. 


Dewey decimal notation applies to any forest: The root of the kth tree in the 
forest is given number k; and if a is the number of any node of degree m, its 
children are numbered a.1,a.2,...,a.m. The Dewey decimal notation satisfies 
many simple mathematical properties, and it is a useful tool in the analysis 
of trees. One example of this is the natural sequential ordering it gives to the 
nodes of an arbitrary tree, analogous to the ordering of sections within this book. 
Section 2.3 precedes Section 2.3.1, and follows Section 2.2.6. 

There is an intimate relation between Dewey decimal notation and the 
notation for indexed variables that we have already been using extensively. If 
F is a forest of trees, we may let F'[1] denote the subtrees of the first tree, so 
that F[1][2] = F[1,2] stands for the subtrees of the second subtree of F[1], and 
F[1,2,1] stands for the first subforest of the latter, and so on. Node a.b.c.d in 
Dewey decimal notation is the parent of F'a, b, c, d]. This notation is an extension 
of ordinary index notation, because the admissible range of each index depends 
on the values in the preceding index positions. 
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2.1 Introduction 
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Fig. 22. The structure of Chapter 2. 


2.3 TREES 315 


Thus, in particular, we see that any rectangular array can be thought of 
as a special case of a tree or forest structure. For example, here are two 
representations of a 3 x 4 matrix: 


Afi,1) Af1,2} Afi,3) Afi = x = 

A[2,1] [2,2] [2,3] A[2, = = = 

A[3.1] 4[,2] 4[3,3] 413, Ji Ji. Ji 
a N oD st = N oD H qt N oD st 
aed NANNAN oD OD OD OD 
LITT WHT VTS 


It is important to observe, however, that this tree structure does not faithfully 
reflect all of the matrix structure; the row relationships appear explicitly in the 
tree but the column relationships do not. 

A forest can, in turn, be regarded as a special case of what is commonly 
called a list structure. The word “list” is being used here in a very technical 
sense, and to distinguish the technical use of the word we will always capitalize it: 
“List.” A List is defined (recursively) as a finite sequence of zero or more atoms 
or Lists. Here “atom” is an undefined concept referring to elements from any 
universe of objects that might be desired, so long as it is possible to distinguish 
an atom from a List. By means of an obvious notational convention involving 
commas and parentheses, we can distinguish between atoms and Lists and we 
can conveniently display the ordering within a List. As an example, consider 


L= (a, (b,a,b), (), & (((2)))), (3) 


which is a List with five elements: first the atom a, then the List (b, a,b), then 
the empty List ( ), then the atom c, and finally the List (((2))). The latter List 
consists of the List ((2)), which consists of the List (2), which consists of the 
atom 2. 

The following tree structure corresponds to L: 


M ae 
r a (4) 


H= 4 *—* 


The asterisks in this diagram indicate the definition and appearance of a List, 
as opposed to the appearance of an atom. Index notation applies to Lists as it 
does to forests; for example, L[2] = (b, a,b), and L[2, 2] = a. 

No data is carried in the nodes for the Lists in (4) other than the fact that 
they are Lists. But it is possible to label the nonatomic elements of Lists with 
information, as we have done for trees and other structures; thus 


A= (a: (b,c), d:()) 


316 INFORMATION STRUCTURES 2.3 
would correspond to a tree that we can draw as follows: 
* 
Z N 
ax dx 
Z N 

b c 

The big difference between Lists and trees is that Lists may overlap (that 


is, sub-Lists need not be disjoint) and they may even be recursive (may contain 
themselves). The List 


M = (M) (5) 
corresponds to no tree structure, nor does the List 
N = (a:M,b:M,c, N). (6) 


(In these examples, capital letters refer to Lists, lowercase letters to labels and 
atoms.) We might diagram (5) and (6) as follows, using an asterisk to denote 
each place where a List is defined: 


*[N] 
A IS 
ax[M] bM] c [N] (7) 


Actually, Lists are not so complicated as the examples above might indicate. 
They are, in essence, a rather simple generalization of the linear lists that we 
have considered in Section 2.2, with the additional proviso that the elements of 
linear Lists may be link variables that point to other linear Lists (and possibly 
to themselves). 


Summary: Four closely related kinds of information structures — trees, for- 
ests, binary trees, and Lists — arise from many sources, and they are therefore 
important in computer algorithms. We have seen various ways to diagram these 
structures, and we have considered some terminology and notations that are 
useful in talking about them. The following sections develop these ideas in 
greater detail. 


EXERCISES 

1. [18] How many different trees are there with three nodes, A, B, and C? 

2. [20] How many different oriented trees are there with three nodes, A, B, and C? 

3. [M20] Prove rigorously from the definitions that for every node X in a tree there is 
a unique path up to the root, namely a unique sequence of k > 1 nodes X1, Xo,..., Xx 
such that X1 is the root of the tree, X, = X, and X; is the parent of Xj+ı for 1 <j < k. 
(This proof will be typical of the proofs of nearly all the elementary facts about tree 
structures.) Hint: Use induction on the number of nodes in the tree. 

4. [01] True or false: In a conventional tree diagram (root at the top), if node X has 


a higher level number than node Y, then node X appears lower in the diagram than 
node Y. 
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5. [02] If node A has three siblings and B is the parent of A, what is the degree 
of B? 


6. [21] Define the statement “X is an mth cousin of Y, n times removed” as a 
meaningful relation between nodes X and Y of a tree, by analogy with family trees, if 
m >0Oand n > 0. (See a dictionary for the meaning of these terms in regard to family 
trees.) 


7. [23] Extend the definition given in the previous exercise to all m > —1 and to all 
integers n > —(m-+ 1) in such a way that for any two nodes X and Y of a tree there 
are unique m and n such that X is an mth cousin of Y, n times removed. 

8. [03] What binary tree is not a tree? 

9. [00] In the two binary trees of (1), which node is the root (B or A)? 

10. [M20] A collection of nonempty sets is said to be nested if, given any pair X, Y 
of the sets, either X C Y or X DY or X and Y are disjoint. (In other words, X N Y 


is either X, Y, or Ø.) Figure 20(a) indicates that any tree corresponds to a collection 
of nested sets; conversely, does every such collection correspond to a tree? 


11. [HM32] Extend the definition of tree to infinite trees by considering collections of 
nested sets as in exercise 10. Can the concepts of level, degree, parent, and child be 
defined for each node of an infinite tree? Give examples of nested sets of real numbers 
that correspond to a tree in which 

a) every node has uncountable degree and there are infinitely many levels; 

b) there are nodes with uncountable level; 

c) every node has degree at least 2 and there are uncountably many levels. 


12. [M23] Under what conditions does a partially ordered set correspond to an un- 
ordered tree or forest? (Partially ordered sets are defined in Section 2.2.3.) 


13. [10] Suppose that node X is numbered a1.a2. ---.a, in the Dewey decimal system; 
what are the Dewey numbers of the nodes in the path from X to the root (see 
exercise 3)? 


14. [M22] Let S be any nonempty set of elements having the form “1.a1.---.ak”, 
where k > 0 and aj,...,ax are positive integers. Show that S specifies a tree when it 
is finite and satisfies the following condition: “If a.m is in the set, then so is a.(m-—1) 
ifm > 1, or a if m = 1.” (This condition is clearly satisfied in the Dewey decimal 
notation for a tree; therefore it is another way to characterize tree structure.) 


15. [20] Invent a notation for the nodes of binary trees, analogous to the Dewey 
decimal notation for nodes of trees. 


16. [20] Draw trees analogous to Fig. 21 corresponding to the arithmetic expressions 
(a) 2(a — b/c); (b) a+ b + 5c. 

17. [01] If Z stands for Fig. 19 regarded as a forest, what node is parent(Z[1, 2, 2])? 
18. [08] In List (3), what is L[5,1,1]? What is L[3, 1]? 

19. [15] Draw a List diagram analogous to (7) for the List L = (a, (L)). What is L[2] 
in this List? What is L[2, 1,1]? 

20. [M21] Define a 0-2-tree as a tree in which each node has exactly zero or two 
children. (Formally, a 0-2-tree consists of a single node, called its root, plus 0 or 2 
disjoint 0-2-trees.) Show that every 0-2-tree has an odd number of nodes; and give a 


one-to-one correspondence between binary trees with n nodes and (ordered) 0-2-trees 
with 2n + 1 nodes. 
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21. [M22] Ifa tree has nı nodes of degree 1, nz nodes of degree 2, ..., and Nnm nodes 
of degree m, how many terminal nodes does it have? 

22. [21] Standard European paper sizes AO, Al, A2,..., An, ... are rectangles whose 
sides are in the ratio v2 to 1 and whose areas are 27” square meters. Therefore if we 
cut a sheet of An paper in half, we get two sheets of A(n + 1) paper. Use this principle 
to design a graphic representation of binary trees, and illustrate your idea by drawing 
the representation of 2.3.1—(1) below. 


2.3.1. Traversing Binary Trees 


It is important to acquire a good understanding of the properties of binary 
trees before making further investigations of trees, since general trees are usually 
represented in terms of some equivalent binary tree inside a computer. 

We have defined a binary tree as a finite set of nodes that either is empty, 
or consists of a root together with two binary trees. This definition suggests a 
natural way to represent binary trees within a computer: We can have two links, 
LLINK and RLINK, within each node, and a link variable T that is a “pointer to 
the tree.” If the tree is empty, T = A; otherwise T is the address of the root node 
of the tree, and LLINK(T), RLINK(T) are pointers to the left and right subtrees of 
the root, respectively. These rules recursively define the memory representation 
of any binary tree; for example, 


is represented by 


ao 
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This simple and natural memory representation accounts for the special 
importance of binary tree structures. We will see in Section 2.3.2 that general 
trees can conveniently be represented as binary trees. Moreover, many trees 
that arise in applications are themselves inherently binary, so binary trees are of 
interest in their own right. 
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There are many algorithms for manipulation of tree structures, and one 
idea that occurs repeatedly in these algorithms is the notion of traversing or 
“walking through” a tree. This is a method of examining the nodes of the tree 
systematically so that each node is visited exactly once. A complete traversal 
of the tree gives us a linear arrangement of the nodes, and many algorithms are 
facilitated if we can talk about the “next” node following or preceding a given 
node in such a sequence. 

Three principal ways may be used to traverse a binary tree: We can visit 
the nodes in preorder, inorder, or postorder. These three methods are defined 
recursively. When the binary tree is empty, it is “traversed” by doing nothing; 
otherwise the traversal proceeds in three steps: 


Preorder traversal Inorder traversal 
Visit the root Traverse the left subtree 
Traverse the left subtree Visit the root 
Traverse the right subtree Traverse the right subtree 


Postorder traversal 


Traverse the left subtree 
Traverse the right subtree 
Visit the root 


If we apply these definitions to the binary tree of (1) and (2), we find that the 
nodes in preorder are 
ABDCEGFH J. (3) 


(First comes the root A, then comes the left subtree 


(B) 
©) 


in preorder, and finally we traverse the right subtree in preorder.) For inorder we 
visit the root between visits to the nodes of each subtree, essentially as though 
the nodes were “projected” down onto a single horizontal line, and this gives the 
sequence 


DBA EGCH F J. (4) 
The postorder for the nodes of this binary tree is, similarly, 
DBGEHIJIFC A (5) 


We will see that these three ways of arranging the nodes of a binary tree into 
a sequence are extremely important, as they are intimately connected with most 
of the computer methods for dealing with trees. The names preorder, inorder, 
and postorder come, of course, from the relative position of the root with respect 
to its subtrees. In many applications of binary trees, there is symmetry between 
the meanings of left subtrees and right subtrees, and in such cases the term 
symmetric order is used as a synonym for inorder. Inorder, which puts the root 
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Fig. 23. Algorithm T for inorder traversal. 


in the middle, is essentially symmetric between left and right: If the binary tree 
is reflected about a vertical axis, the symmetric order is simply reversed. 

A recursively stated definition, such as the one just given for the three basic 
orders, must be reworked in order to make it directly applicable to computer 
implementation. General methods for doing this are discussed in Chapter 8; we 
usually make use of an auxiliary stack, as in the following algorithm: 


Algorithm T (Traverse binary tree in inorder). Let T be a pointer to a binary 
tree having a representation as in (2); this algorithm visits all the nodes of the 
binary tree in inorder, making use of an auxiliary stack A. 


T1. [Initialize.] Set stack A empty, and set the link variable P + T. 
T2. [P = 4?] If P = A, go to step T4. 


T3. [Stack < P.] (Now P points to a nonempty binary tree that is to be 
traversed.) Set A < P; that is, push the value of P onto stack A. (See 
Section 2.2.1.) Then set P + LLINK(P) and return to step T2. 


T4. [P < Stack.] If stack A is empty, the algorithm terminates; otherwise set 
P<A. 


T5. [Visit P.] Visit NODE(P). Then set P + RLINK(P) and return to step T2. J 


In the final step of this algorithm, the word “visit” means that we do 
whatever activity is intended as the tree is being traversed. Algorithm T runs like 
a coroutine with respect to this other activity: The main program activates the 
coroutine whenever it wants P to move from one node to its inorder successor. 
Of course, since this coroutine calls the main routine in only one place, it is 
not much different from a subroutine (see Section 1.4.2). Algorithm T assumes 
that the external activity deletes neither NODE(P) nor any of its ancestors from 
the tree. 

The reader should now attempt to play through Algorithm T using the 
binary tree (2) as a test case, in order to see the reasons behind the procedure. 
When we get to step T3, we want to traverse the binary tree whose root is 
indicated by pointer P. The idea is to save P on a stack and then to traverse the 
left subtree; when this has been done, we will get to step T4 and will find the 


2.3.1 TRAVERSING BINARY TREES 321 


old value of P on the stack again. After visiting the root, NODE(P), in step T5, 
the remaining job is to traverse the right subtree. 

Algorithm T is typical of many other algorithms that we will see later, so 
it is instructive to look at a formal proof of the remarks made in the preceding 
paragraph. Let us now attempt to prove that Algorithm T traverses a binary tree 
of n nodes in inorder, by using induction on n. Our goal is readily established if 
we can prove a slightly more general result: 


Starting at step T2 with P a pointer to a binary tree of n nodes and with 
the stack A containing A[1] ...A[m] for some m > 0, the procedure of steps 
T2-T5 will traverse the binary tree in question, in inorder, and will then 
arrive at step T4 with stack A returned to its original value A[1]...A[m]. 


This statement is obviously true when n = 0, because of step T2. If n > 0, 
let Po be the value of P upon entry to step T2. Since Po Æ A, we will perform 
step T3, which means that stack A is changed to A[1]...A[m]Po and P is set 
to LLINK(P9). Now the left subtree has fewer than n nodes, so by induction we 
will traverse the left subtree in inorder and will ultimately arrive at step T4 with 
A[1]...A[m] Po on the stack. Step T4 returns the stack to A[1]...A[m] and 
sets P + Po. Step T5 now visits NODE (Po) and sets P + RLINK (Po). Now the 
right subtree has fewer than n nodes, so by induction we will traverse the right 
subtree in inorder and arrive at step T4 as required. The tree has been traversed 
in inorder, by the definition of that order. This completes the proof. 

An almost identical algorithm may be formulated that traverses binary trees 
in preorder (see exercise 12). It is slightly more difficult to achieve the traversal 
in postorder (see exercise 13), and for this reason postorder is not as important 
for binary trees as the others are. 

It is convenient to define a new notation for the successors and predecessors 
of nodes in these various orders. If P points to a node of a binary tree, let 


Px = address of successor of NODE(P) in preorder; 

Ps = address of successor of NODE(P) in inorder; 

Pt = address of successor of NODE(P) in postorder; 
xP = address of predecessor of NODE (P) in preorder; 
$P = address of predecessor of NODE (P) in inorder; 
#P = address of predecessor of NODE(P) in postorder. 


If there is no such successor or predecessor of NODE(P), the value LOC(T) is 
generally used, where T is an external pointer to the tree in question. We have 
*(Px) = (*P)* = P, $(Ps) = (sP)s = P, and 4(Pi) = (#P)# = P. As an example of 
this notation, let INFO(P) be the letter shown in NODE(P) in the tree (2); then 
if P points to the root, we have INFO(P) = A, INFO(Px) = B, INFO(Ps) = E, 
INFO(sP) = B, INFO(#P) = C, and Pi = +P = LOC(T). 

At this point the reader will perhaps experience a feeling of insecurity about 
the intuitive meanings of Px, Ps, etc. As we proceed further, the ideas will 
gradually become clearer; exercise 16 at the end of this section may also be of 
help. The “$” in “Ps” is meant to suggest the letter S, for “symmetric order.” 
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There is an important alternative to the memory representation of binary 
trees given in (2), which is somewhat analogous to the difference between circular 
lists and straight one-way lists. Notice that there are more null links than other 
pointers in the tree (2), and indeed this is true of any binary tree represented by 
the conventional method (see exercise 14). But we don’t really need to waste all 
that memory space. For example, we could store two “tag” indicators with each 
node, which would tell in just two bits of memory whether or not the LLINK or 
RLINK, or both, are null; the memory space for terminal links could then be used 
for other purposes. 

An ingenious use of this extra space has been suggested by A. J. Perlis 
and C. Thornton, who devised the so-called threaded tree representation. In this 
method, terminal links are replaced by “threads” to other parts of the tree, as 
an aid to traversal. The threaded tree equivalent to (2) is 


Here dotted lines represent the “threads,” which always go to a higher node of the 
tree. Every node now has two links: Some nodes, like C, have two ordinary links 
to left and right subtrees; other nodes, like H, have two thread links; and some 
nodes have one link of each type. The special threads emanating from D and J 
will be explained later. They appear in the “leftmost” and “rightmost” nodes. 

In the memory representation of a threaded binary tree it is necessary to 
distinguish between the dotted and solid links; this can be done as suggested 
above by two additional one-bit fields in each node, LTAG and RTAG. The threaded 
representation may be defined precisely as follows: 


Unthreaded representation Threaded representation 
LLINK(P) = A LTAG(P) = 1, LLINK(P) = $P 
LLINK(P) =Q4A LTAG(P) =0, LLINK(P) =Q 
RLINK(P) = A RTAG(P) = 1, RLINK(P) = Ps 
RLINK(P) =Q4A RTAG(P) =0, RLINK(P) =Q 


According to this definition, each new thread link points directly to the 
predecessor or successor of the node in question, in symmetric order (inorder). 
Figure 24 illustrates the general orientation of thread links in any binary tree. 

In some algorithms it can be guaranteed that the root of any subtree always 
will appear in a lower memory location than the other nodes of the subtree. 
Then LTAG(P) will be 1 if and only if LLINK(P) < P, so LTAG will be redundant. 
The RTAG bit will be redundant for the same reason. 
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General k 


General k 


Fig. 24. General orientation of left and right thread links in a threaded binary tree. 
Wavy lines indicate links or threads to other parts of the tree. 


The great advantage of threaded trees is that traversal algorithms become 
simpler. For example, the following algorithm calculates Ps, given P: 


Algorithm S (Symmetric (inorder) successor in a threaded binary tree). If P 
points to a node of a threaded binary tree, this algorithm sets Q + Ps. 


S1. [RLINK(P) a thread?] Set Q 4 RLINK(P). If RTAG(P) = 1, terminate the 
algorithm. 


S2. [Search to left.] If LTAG(Q) = 0, set Q + LLINK(Q) and repeat this step. 
Otherwise the algorithm terminates. Į 


Notice that no stack is needed here to accomplish what was done using a 
stack in Algorithm T. In fact, the ordinary representation (2) makes it impossible 
to find P$ efficiently, given only the address of a random point P in the tree. Since 
no links point upward in an unthreaded representation, there is no clue to what 
nodes are above a given node, unless we retain a history of how we reached that 
point. The stack in Algorithm T provides the necessary history when threads 
are absent. 

We claim that Algorithm S is “efficient,” although this property is not 
immediately obvious, since step S2 can be executed any number of times. In 
view of the loop in step $2, would it perhaps be faster to use a stack after all, 
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as Algorithm T does? To investigate this question, we will consider the average 
number of times that step S2 must be performed if P is a “random” point in 
the tree; or what is the same, we will determine the total number of times that 
step S2 is performed if Algorithm S is used repeatedly to traverse an entire tree. 

At the same time as this analysis is being carried out, it will be instructive 
to study complete programs for both Algorithms S and T. As usual, we should 
be careful to set all of our algorithms up so that they work properly with empty 
binary trees; and if T is the pointer to the tree, we would like to have LOC(T) x 
and LOC(T)s be the first nodes in preorder or symmetric order, respectively. For 
threaded trees, it turns out that things will work nicely if NODE(LOC(T)) is made 
into a “list head” for the tree, with 


LLINK (HEAD) = T, LTAG(HEAD) = 0, (8) 
RLINK (HEAD) = HEAD, RTAG(HEAD) = 0. 


(Here HEAD denotes LOC(T), the address of the list head.) An empty threaded 
tree will satisfy the conditions 


LLINK(HEAD) = HEAD, LTAG(HEAD) = 1. (9) 


The tree grows by having nodes inserted to the left of the list head. (These 
initial conditions are primarily dictated by the algorithm to compute Px, which 
appears in exercise 17.) In accordance with these conventions, the computer 
representation for the binary tree (1), as a threaded tree, is 


List head 


With these preliminaries out of the way, we are now ready to consider MIX 
versions of Algorithms S and T. The following programs assume that binary tree 
nodes have the two-word form 


LTAG| LLINK INFO1 
RTAG| RLINK INFO2 


In an unthreaded tree, LTAG and RTAG will always be “+” and terminal links will 
be represented by zero. In a threaded tree, we will use “+” for tags that are 0 
and “—” for tags that are 1. The abbreviations LLINKT and RLINKT will be used 
to stand for the combined LTAG-LLINK and RTAG-RLINK fields, respectively. 
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~ The two tag bits occupy otherwise-unused sign positions of a MIX word, so 

{£ they cost nothing in memory space. Similarly, with the MMIX computer we 
will be able to use the least significant bits of link fields as tag bits that come 
“for free,” because pointer values will generally be even, and because MMIX will 
make it easy to ignore the low-order bits when addressing memory. 


The following two programs traverse a binary tree in symmetric order (that 
is, inorder), jumping to location VISIT periodically with index register 5 pointing 
to the node that is currently of interest. 


Program T. In this implementation of Algorithm T, the stack is kept in loca- 
tions A+1,A+2,..., A + MAX; rI6 is the stack pointer and rl5 = P. OVERFLOW 
occurs if the stack grows too large. The program has been rearranged slightly 
from Algorithm T (step T2 appears thrice), so that the test for an empty stack 
need not be made when going directly from T3 to T2 to T4. 


01 LLINK EQU 1:2 
02 RLINK EQU 1:2 


03 T1 LD5 HEAD(LLINK) 1 Ti. Initialize. Set P + T. 

04 T2A J5Z DONE 1 Stop if P = A. 

05 ENT6 O 1 

06 T3 DEC6 MAX n T3. Stack < P. 

07 J6NN OVERFLOW n Has that stack reached capacity? 
08 INC6 MAX+1 n If not, increase the stack pointer. 
09 ST5 A,6 n Store P in the stack. 

10 LD5 O,5(LLINK) n P+ LLINK(P). 

11 T2B J5NZ T3 n To T3ifP Æ. 

12 T4 LD5 A,6 n T4. P< Stack. 

13 DEC6 1 n Decrease the stack pointer. 

14 T5 JMP VISIT n T5. Visit P. 

15 LD5 1,5(RLINK) n P<- RLINK(P). 

16 T2C J5NZ T3 n T2. P= A? 

17 J6NZ T4 a Test if the stack is empty. 

18 DONE ... I 


Program S. Algorithm S has been augmented with initialization and termina- 
tion conditions to make this program comparable to Program T. 


01 LLINKT EQU 0:2 
02 RLINKT EQU 0:2 


03 SO ENT5 HEAD 1 S0. Initialize. Set P 4+ HEAD. 

04 JMP 2F 1 

05 $3 JMP VISIT n 53. Visit P. 

06 St LD5N 1,5(RLINKT) n 5S1. RLINK(P) a thread? 

07 J5NN 1F n Jump if RTAG(P) = 1. 

0s ENN6 0,5 n—a Otherwise set Q + RLINK (P). 
09 S2 ENT5 0,6 n S2. Search to left. Set P + Q. 
10 2H LD6 0,5(LLINKT) n+1 Q< LLINKT(P). 

11 J6P 52 n+1 IfLTAG(P) = 0, repeat. 

12 1H ENT6 -HEAD,5 n+1 

18 J6NZ 53 n+1 Visit unless P = HEAD. J 
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An analysis of the running time appears with the code above. These quan- 
tities are easy to determine, using Kirchhoff’s law and the facts that 


i) in Program T, the number of insertions onto the stack must equal the number 
of deletions; 
ii) in Program S, the LLINK and RLINK of each node are examined precisely 
once; 
iii) the number of “visits” is the number of nodes in the tree. 


The analysis tells us Program T takes 15n+a+4 units of time, and Program S 
takes 11n — a + 7 units, where n is the number of nodes in the tree and a is the 
number of terminal right links (nodes with no right subtree). The quantity a 
can be as low as 1, assuming that n 4 0, and it can be as high as n. If left and 
right are symmetrical, the average value of a is (n + 1)/2, as a consequence of 
facts proved in exercise 14. 

The principal conclusions we may reach on the basis of this analysis are: 


i) Step S2 of Algorithm S is performed only once on the average per execution 
of that algorithm, if P is a random node of the tree. 

ii) Traversal is slightly faster for threaded trees, because it requires no stack 
manipulation. 

iii) Algorithm T needs more memory space than Algorithm S because of the 
auxiliary stack required. In Program T we kept the stack in consecutive 
memory locations; therefore we needed to put an arbitrary bound on its 
size. It would be very embarrassing if this bound were exceeded, so it must 
be set reasonably large (see exercise 10); thus the memory requirement 
of Program T is significantly more than Program S. Not infrequently a 
complex computer application will be independently traversing several trees 
at once, and a separate stack will be needed for each tree under Program T. 
This suggests that Program T might use linked allocation for its stack (see 
exercise 20); its execution time then becomes 30n + a+ 4 units, roughly 
twice as slow as before, although the traversal speed may not be terribly 
important when the execution time for the other coroutine is added in. Still 
another alternative is to keep the stack links within the tree itself in a tricky 
way, as discussed in exercise 21. 

iv) Algorithm S is, of course, more general than Algorithm T, since it allows us 
to go from P to P$ when we are not necessarily traversing the entire binary 
tree. 


So a threaded binary tree is decidedly superior to an unthreaded one, with 
respect to traversal. These advantages are offset in some applications by the 
slightly increased time needed to insert and delete nodes in a threaded tree. It 
is also sometimes possible to save memory space by “sharing” common subtrees 
with an unthreaded representation, while threaded trees require adherence to a 
strict tree structure with no overlapping of subtrees. 

Thread links can also be used to compute Px, $P, and #P with efficiency 
comparable to that of Algorithm S. The functions *P and P# are slightly harder 
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to compute, just as they are for unthreaded tree representations. The reader is 
urged to work exercise 17. 

Most of the usefulness of threaded trees would disappear if it were hard to 
set up the thread links in the first place. What makes the idea really work is that 
threaded trees grow almost as easily as ordinary ones do. We have the following 
algorithm: 


Algorithm I (Insertion into a threaded binary tree). This algorithm attaches 
a single node, NODE(Q), as the right subtree of NODE (P), if the right subtree is 
empty (that is, if RTAG(P) = 1); otherwise it inserts NODE(Q) between NODE (P) 
and NODE(RLINK(P)), making the latter node the right child of NODE(Q). The 
binary tree in which the insertion takes place is assumed to be threaded as in 
(10); for a modification, see exercise 23. 


T1. [Adjust tags and links.] Set RLINK(Q) < RLINK(P), RTAG(Q) + RTAG(P), 
RLINK(P) «+ Q, RTAG(P) + 0, LLINK(Q) + P, LTAG(Q) + 1. 


I2. [Was RLINK(P) a thread?) If RTAG(Q) = 0, set LLINK(Qs) + Q. (Here Qs is 
determined by Algorithm S, which will work properly even though LLINK (Qs) 
now points to NODE(P) instead of NODE(Q). This step is necessary only when 
inserting into the midst of a threaded tree instead of merely inserting a new 
leaf.) J 


By reversing the roles of left and right (in particular, by replacing Qs by $Q 
in step I2), we obtain an algorithm that inserts to the left in a similar way. 

Our discussion of threaded binary trees so far has made use of thread links 
both to the left and to the right. There is an important middle ground between 
the completely unthreaded and completely threaded methods of representation: 
A right-threaded binary tree combines the two approaches by making use of 
threaded RLINKs, while representing empty left subtrees by LLINK = A. (Simi- 
larly, a left-threaded binary tree threads only the null LLINKs.) Algorithm S does 
not make essential use of threaded LLINKs; if we change the test “LTAG = 0” in 
step S2 to “LLINK # A”, we obtain an algorithm for traversing right-threaded 
binary trees in symmetric order. Program S works without change in the right- 
threaded case. A great many applications of binary tree structures require only 
a left-to-right traversal of trees using the functions Ps and/or Px, and for these 
applications there is no need to thread the LLINKs. We have described threading 
in both the left and right directions in order to indicate the symmetry and 
possibilities of the situation, but in practice one-sided threading is much more 
common. 

Let us now consider an important property of binary trees, and its con- 
nection to traversal. Two binary trees T and T’ are said to be similar if they 
have the same structure; formally, this means that (a) they are both empty, or 
(b) they are both nonempty and their left and right subtrees are respectively 
similar. Similarity means, informally, that the diagrams of T and T’ have the 
same “shape.” Another way to phrase similarity is to say that there is a one-to- 
one correspondence between the nodes of T and T’ that preserves the structure: 
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If nodes u; and ug in T correspond respectively to ui and u% in T’, then u; is in 
the left subtree of uz if and only if u) is in the left subtree of u5, and the same 
is true for right subtrees. 

The binary trees T and T” are said to be equivalent if they are similar and if 
corresponding nodes contain the same information. Formally, let info(w) denote 
the information contained in a node u; the trees are equivalent if and only if 
(a) they are both empty, or (b) they are both nonempty and info(root(T’)) = 
info (root (T’ ’)) and their left and right subtrees are respectively equivalent. 

As examples of these definitions, consider the four binary trees 


in which the first two are dissimilar. The second, third, and fourth are similar 
and, in fact, the second and fourth are equivalent. 

Some computer applications involving tree structures require an algorithm 
to decide whether two binary trees are similar or equivalent. The following 
theorem is useful in this regard: 


Theorem A. Let the nodes of binary trees T and T’ be respectively 


È / / 
U1,UQ,-.+,Un and Uy, Ug, +++, Uns 


in preorder. For any node u let 


I(u) =1 ifu has a nonempty left subtree, I(u) =0 otherwise; Gi) 
r(u)=1 if u has a nonempty right subtree, r(u) =0 otherwise. 


Then T and T” are similar if and only if n =n’ and 


I(uj) = I(uj), r(uj) = r(u;) forl<j<n. (12) 


Moreover, T and T’ are equivalent if and only if in addition we have 
info(u;) =info(u;)  forl<j<n. (13) 


Notice that l and r are the complements of the LTAG and RTAG bits in a 
threaded tree. This theorem characterizes any binary tree structure in terms of 
two sequences of Os and 1s. 


Proof. It is clear that the condition for equivalence of binary trees will follow 
immediately if we prove the condition for similarity; furthermore the conditions 
n = n’ and (12) are certainly necessary, since corresponding nodes of similar 
trees must have the same position in preorder. Therefore it suffices to prove that 
the conditions (12) and n = n’ are sufficient to guarantee the similarity of T 
and T”. The proof is by induction on n, using the following auxiliary result: 
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Lemma P. Let the nodes of a nonempty binary tree be u1, U2, ..., Un in preorder, 
and let f(u) = l(u)+r(u)— 1. Then 


f(ui)+flu2)+ --+flun)=—1, and f(ui)+---+f(ug)>0, 1<k<n. (14) 


Proof. The result is clear for n = 1. If n > 1, the binary tree consists of its 
root uw; and further nodes. If f(u1) = 0, then either the left subtree or the right 
subtree is empty, so the condition is obviously true by induction. If f(u) = 1, 
let the left subtree have n; nodes; by induction we have 


fa) ++ flux) >0 forl<k<m, f(ui)+---+f(un41)=0, (15) 


and the condition (14) is again evident. J 


(For other theorems analogous to Lemma P, see the discussion of Polish 
notation in Chapter 10.) 

To complete the proof of Theorem A, we note that the theorem is clearly 
true when n = 0. If n > 0, the definition of preorder implies that u; and uj are 
the respective roots of their trees, and there are integers n, and n; (the sizes of 
the left subtrees) such that 


U2,- , Un 41 and TE Uni 41 are the left subtrees of T and T’; 
Un 420+ +- Un and Un 42> ...,u,, are the right subtrees of T and T”. 


The proof by induction will be complete if we can show n, = n;. There are three 
cases: 


if l(u1) = 0, then n, =0 =n); 

if l(u1) = 1, r(u1) = 0, then n =n — 1 = ni; 

if ((u1) = r(u1) = 1, then by Lemma P we can find the least k > 0 such 
that f(u1) +---+ f(ug) = 0; and n, = k — 1 = n; (see (15)). I 


As a consequence of Theorem A, we can test two threaded binary trees for 
equivalence or similarity by simply traversing them in preorder and checking 
the INFO and TAG fields. Some interesting extensions of Theorem A have been 
obtained by A. J. Blikle, Bull. de l’Acad. Polonaise des Sciences, Série des 
Sciences Math., Astr., Phys., 14 (1966), 203-208; he considered an infinite class 
of possible traversal orders, only six of which (including preorder) were called 
“addressless” because of their simple properties. 

We conclude this section by giving a typical, yet basic, algorithm for binary 
trees, one that makes a copy of a binary tree into different memory locations. 


Algorithm C (Copy a binary tree). Let HEAD be the address of the list head 
of a binary tree T; thus, T is the left subtree of HEAD, reached via LLINK (HEAD). 
Let NODE(U) be a node with an empty left subtree. This algorithm makes a copy 
of T and the copy becomes the left subtree of NODE(U). In particular, if NODE (U) 
is the list head of an empty binary tree, this algorithm changes the empty tree 
into a copy of T. 


C1. [Initialize.] Set P + HEAD, Q + U. Go to C4. 
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C2. [Anything to right?] If NODE(P) has a nonempty right subtree, set R = 
AVAIL, and attach NODE(R) to the right of NODE(Q). (At the beginning of 
step C2, the right subtree of NODE(Q) was empty.) 


C3. [Copy INFO.] Set INFO(Q) + INFO(P). (Here INFO denotes all parts of the 

node that are to be copied, except for the links.) 

C4. [Anything to left?] If NODE(P) has a nonempty left subtree, set R <= AVAIL, 

and attach NODE(R) to the left of NODE(Q). (At the beginning of step C4, 

the left subtree of NODE(Q) was empty.) 

C5. [Advance.] Set P + Px, Q + Qx. 

C6. [Test if complete.] If P = HEAD (or equivalently if Q = RLINK(U), assuming 
that NODE(U) has a nonempty right subtree), the algorithm terminates; 
otherwise go to step C2. J 


This simple algorithm shows a typical application of tree traversal. The 
description here applies to threaded, unthreaded, or partially threaded trees. 
Step C5 requires the calculation of preorder successors P» and Q*; for unthreaded 
trees, this generally is done with an auxiliary stack. A proof of the validity 
of Algorithm C appears in exercise 29; a MIX program corresponding to this 
algorithm in the case of a right-threaded binary tree appears in exercise 2.3.2-13. 
For threaded trees, the “attaching” in steps C2 and C4 is done using Algorithm I. 

The exercises that follow include quite a few topics of interest relating to 
the material of this section. 


Binary or dichotomous systems, although regulated by a principle, 
are among the most artificial arrangements 
that have ever been invented. 


— WILLIAM SWAINSON, A Treatise on the Geography and 
Classification of Animals (1835) 


EXERCISES 
1. [01] In the binary tree (2), let INFO(P) denote the letter stored in NODE(P). What 
is INFO(LLINK(RLINK (RLINK(T))))? D 


2. [11] List the nodes of the binary tree @ % in (a) preorder; (b) symmetric 
order; (c) postorder. dOAO 


3. [20] Is the following statement true or false? “The terminal nodes of a binary 
tree occur in the same relative position in preorder, inorder, and postorder.” 


4. [20] The text defines three basic orders for traversing a binary tree; another 
alternative would be to proceed in three steps as follows: 


a) Visit the root, 
b) traverse the right subtree, 
c) traverse the left subtree, 


using the same rule recursively on all nonempty subtrees. Does this new order bear 
any simple relation to the three orders already discussed? 
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5. [22] The nodes of a binary tree may be identified by a sequence of zeros and 
ones, in a notation analogous to “Dewey decimal notation” for trees, as follows: The 
root (if present) is represented by the sequence “1”. Roots (if present) of the left 
and right subtrees of the node represented by a are respectively represented by a0 
and al. For example, the node H in (1) would have the representation “1110”. (See 
exercise 2.3-15.) 

Show that preorder, inorder, and postorder can be described conveniently in terms 
of this notation. 


6. [M22] Suppose that a binary tree has n nodes that are u1 u2...tUn in preorder 
and Up, Ups ..-Up,, in inorder. Show that the permutation p1p2...pn can be obtained 
by passing 12...n through a stack, in the sense of exercise 2.2.1-2. Conversely, show 
that any permutation pip2...pn obtainable with a stack corresponds to some binary 
tree in this way. 


7. [22] Show that if we are given the preorder and the inorder of the nodes of a 
binary tree, the binary tree structure may be constructed. (Assume that the nodes are 
distinct.) Does the same result hold true if we are given the preorder and postorder, 
instead of preorder and inorder? Or if we are given the inorder and postorder? 


8. [20] Find all binary trees whose nodes appear in exactly the same sequence in 
both (a) preorder and inorder; (b) preorder and postorder; (c) inorder and postorder. 
(As in the previous exercise, we assume that the nodes have distinct labels.) 


9. [M20] When a binary tree having n nodes is traversed using Algorithm T, state 
how many times each of steps T1, T2, T3, T4, and T5 is performed (as a function of n). 


> 10. [20] What is the largest number of entries that can be in the stack at once, 
during the execution of Algorithm T, if the binary tree has n nodes? (The answer 
to this question is very important for storage allocation, if the stack is being stored 
consecutively.) 


11. [HM41] Analyze the average value of the largest stack size occurring during the 
execution of Algorithm T as a function of n, given that all binary trees with n nodes 
are considered equally probable. 


12. [22] Design an algorithm analogous to Algorithm T that traverses a binary tree 
in preorder, and prove that your algorithm is correct. 


> 13. [24] Design an algorithm analogous to Algorithm T that traverses a binary tree 
in postorder. 


14. [20] Show that if a binary tree with n nodes is represented as in (2), the total 
number of A links in the representation can be expressed as a simple function of n; this 
quantity does not depend on the shape of the tree. 


15. [15] In a threaded-tree representation like (10), each node except the list head has 
exactly one link pointing to it from above, namely the link from its parent. Some of the 
nodes also have links pointing to them from below; for example, the node containing C 
has two pointers coming up from below, while node E has just one. Is there any 
simple connection between the number of links pointing to a node and some other 
basic property of that node? (We need to know how many links point to a given node 
when we are changing the tree structure.) 


> 16. [22] The diagrams in Fig. 24 help to provide an intuitive characterization of the 
position of NODE (Qs) in a binary tree, in terms of the structure near NODE (Q): If NODE (Q) 
has a nonempty right subtree, consider Q = $P, Qs = P in the upper diagrams; NODE (Q$) 
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is the “leftmost” node of that right subtree. If NODE(Q) has an empty right subtree, 
consider Q = P in the lower diagrams; NODE(Q$) is located by proceeding upward in the 
tree until after the first upward step to the right. 

Give a similar “intuitive” rule for finding the position of NODE(Q*) in a binary tree 
in terms of the structure near NODE(Q). 


17. [22] Give an algorithm analogous to Algorithm S for determining Px in a threaded 
binary tree. Assume that the tree has a list head as in (8), (9), and (10). 


18. [24] Many algorithms dealing with trees like to visit each node twice instead of 
once, using a combination of preorder and inorder that we might call double order. 
Traversal of a binary tree in double order is defined as follows: If the binary tree is 
empty, do nothing; otherwise 


a) visit the root, for the first time; 

b) traverse the left subtree, in double order; 
c) visit the root, for the second time; 

d) traverse the right subtree, in double order. 


For example, traversal of (1) in double order gives the sequence 
A; Bı Dı D2 B2A2C1 E1 E2G1G2C2 Fi H1 H2 Fo Ji Jo, 


where Aı means that A is being visited for the first time. 

If P points to a node of the tree and if d = 1 or 2, define (P,d)“ = (Q,e) if the 
next step in double order after visiting NODE(P) the dth time is to visit NODE(Q) the eth 
time; or, if (P, d) is the last step in double order, we write (P, d) = (HEAD, 2), where 
HEAD is the address of the list head. We also define (HEAD, 1)^ as the first step in double 
order. 

Design an algorithm analogous to Algorithm T that traverses a binary tree in 
double order, and also design an algorithm analogous to Algorithm S that computes 
(P, d). Discuss the relation between these algorithms and exercises 12 and 17. 


19. [27] Design an algorithm analogous to Algorithm S for the calculation of P# in 
(a) a right-threaded binary tree; (b) a fully threaded binary tree. If possible, the 
average running time of your algorithm should be at most a small constant, when P is 
a random node of the tree. 


20. [23] Modify Program T so that it keeps the stack in a linked list, not in consecutive 
memory locations. 


21. [33] Design an algorithm that traverses an unthreaded binary tree in inorder 
without using any auailiary stack. It is permissible to alter the LLINK and RLINK fields 
of the tree nodes in any manner whatsoever during the traversal, subject only to the 
condition that the binary tree should have the conventional representation illustrated 
in (2) both before and after your algorithm has traversed the tree. No other bits in the 
tree nodes are available for temporary storage. 


22. [25] Write a MIX program for the algorithm given in exercise 21 and compare its 
execution time to Programs S and T. 


23. [22] Design algorithms analogous to Algorithm I for insertion to the right and 
insertion to the left in a right-threaded binary tree. Assume that the nodes have the 
fields LLINK, RLINK, and RTAG. 


24. [M20] Is Theorem A still valid if the nodes of T and T’ are given in symmetric 
order instead of preorder? 
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25. [M24] Let T be a set of binary trees in which the value of each info field belongs 
to a given set S, where S is linearly ordered by a relation “<” (see exercise 2.2.3-14). 
Given any trees T, T’ in T, let us now define T < T’ if and only if 


i) T is empty; or 
ii) T and T” are not empty, and info(root(T)) < info(root(T’)); or 
iii) T and T’ are not empty, info(root(T)) = info(root(T")), left(T) < left(T’), and 
left(T) is not equivalent to left(T’); or 
iv) T and T’ are not empty, info(root(T’)) = info(root(T”)), left(T) is equivalent to 
left(T’), and right(T) < right(T’). 
Here left(T) and right(T) denote the left and right subtrees of T. Prove that (a) T < T’ 
and T’ < T” implies T < T”; (b) T is equivalent to T” if and only if T < T’ and T’ = T; 
(c) for any T, T’ in T we have either T < T’ or T’ x T. [Thus, if equivalent trees in T 
are regarded as equal, the relation < induces a linear ordering on 7. This ordering has 
many applications (for example, in the simplification of algebraic expressions). When 
S has only one element, so that the “info” of each node is the same, we have the special 
case that equivalence is the same as similarity.] 


26. [M24] Consider the ordering T < T’ defined in the preceding exercise. Prove 
a theorem analogous to Theorem A, giving a necessary and sufficient condition that 
T < T’, and making use of double order as defined in exercise 18. 


27. [28] Design an algorithm that tests two given trees T and T’ to see whether 
T <T',T > T’, or T is equivalent to T’, in terms of the relation defined in exercise 25, 
assuming that both binary trees are right-threaded. Assume that each node has the 
fields LLINK, RLINK, RTAG, INFO; use no auxiliary stack. 


28. [00] After Algorithm C has been used to make a copy of a tree, is the new binary 
tree equivalent to the original, or similar to it? 


29. [M25] Prove as rigorously as possible that Algorithm C is valid. 


30. [22] Design an algorithm that threads an unthreaded tree; for example, it should 
transform (2) into (10). Note: Always use notations like Px and Ps when possible, 
instead of repeating the steps for traversal algorithms like Algorithm T. 


31. [23] Design an algorithm that “erases” a right-threaded binary tree. Your algo- 
rithm should return all of the tree nodes except the list head to the AVAIL list, and it 
should make the list head signify an empty binary tree. Assume that each node has 
the fields LLINK, RLINK, RTAG; use no auxiliary stack. 


32. [21] Suppose that each node of a binary tree has four link fields: LLINK and RLINK, 
which point to left and right subtrees or A, as in an unthreaded tree; SUC and PRED, 
which point to the successor and predecessor of the node in symmetric order. (Thus 
SUC(P) = Ps and PRED (P) = $P. Such a tree contains more information than a threaded 
tree.) Design an algorithm like Algorithm I for insertion into such a tree. 


33. [30] There is more than one way to thread a tree! Consider the following repre- 
sentation, using three fields LTAG, LLINK, RLINK in each node: 


LTAG(P): defined the same as in a threaded binary tree; 
LLINK(P): always equal to Px; 
RLINK(P): defined the same as in an unthreaded binary tree. 


Discuss insertion algorithms for such a representation, and write out the copying 
algorithm, Algorithm C, in detail for this representation. 
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34. [22] Let P point to a node in some binary tree, and let HEAD point to the list head 
of an empty binary tree. Give an algorithm that (i) removes NODE(P) and all of its 
subtrees from whatever tree it was in, and then (ii) attaches NODE(P) and its subtrees 
to NODE(HEAD). Assume that all the binary trees in question are right-threaded, with 
fields LLINK, RTAG, RLINK in each node. 

35. [40] Define a ternary tree (and, more generally, a t-ary tree for any t > 2) ina 
manner analogous to our definition of a binary tree, and explore the topics discussed 
in this section (including topics found in the exercises above) that can be generalized 
to t-ary trees in a meaningful way. 

36. [M23] Exercise 1.2.1-15 shows that lexicographic order extends a well-ordering of 
a set S to a well-ordering of the n-tuples of elements of S. Exercise 25 above shows that 
a linear ordering of the information in tree nodes can be extended to a linear ordering 
of trees, using a similar definition. If the relation < well-orders S, is the extended 
relation of exercise 25 a well-ordering of 7? 

37. [24] (D. Ferguson.) If two computer words are necessary to contain two link fields 
and an INFO field, representation (2) requires 2n words of memory for a tree with n 
nodes. Design a representation scheme for binary trees that uses less space, assuming 
that one link and an INFO field will fit in a single computer word. 


2.3.2. Binary Tree Representation of Trees 


We turn now from binary trees to just plain trees. Let us recall the basic 
differences between trees and binary trees as we have defined them: 
1) A tree always has a root node, so it is never empty; each node of a tree can 
have 0, 1, 2, 3, ... children. 
2) A binary tree can be empty, and each of its nodes can have 0, 1, or 2 children; 
we distinguish between a “left” child and a “right” child. 
Recall also that a forest is an ordered set of zero or more trees. The subtrees 
immediately below any node of a tree form a forest. 
There is a natural way to represent any forest as a binary tree. Consider 
the following forest of two trees: 


(A) D 
© ©) ©) (1) 
®© @ 


The corresponding binary tree is obtained by linking together the children of 
each family and removing vertical links except from a parent to a first child: 
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Then, tilt the diagram 45° clockwise and tweak it slightly, obtaining a binary 
tree: 


Conversely, it is easy to see that any binary tree corresponds to a unique forest 
of trees by reversing the process. 

The transformation from (1) to (3) is extremely important; it is called the 
natural correspondence between forests and binary trees. In particular, it gives 
a correspondence between trees and a special class of binary trees, namely the 
binary trees that have a root but no right subtree. (We might also change our 
viewpoint slightly and let the root of a tree correspond to the list head of a binary 
tree, thus obtaining a one-to-one correspondence between trees with n+ 1 nodes 
and binary trees with n nodes.) 

Let F = (T\,To,...,T,) be a forest of trees. The binary tree B(F) corre- 
sponding to F can be defined rigorously as follows: 

a) If n= 0, B(F) is empty. 

b) If n > 0, the root of B(F) is root(Tı); the left subtree of B(F) is B(Ti1, Ti2, 
..., Tim), where T11, T12, . . ., Tim are the subtrees of root(T;); and the right 
subtree of B(F) is B(To,..., Tn). 

These rules specify the transformation from (1) to (3) precisely. 


It will occasionally be convenient to draw our binary tree diagram as in (2), 
without the 45° rotation. The threaded binary tree corresponding to (1) is 


(compare with Fig. 24, giving the latter a 45° change in orientation). Notice 
that right thread links go from the rightmost child of a family to the parent. 
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Left thread links do not have such a natural interpretation, due to the lack of 
symmetry between left and right. 

The ideas about traversal explored in the previous section can be recast 
in terms of forests (and, therefore, trees). There is no simple analog of the 
inorder sequence, since there is no obvious place to insert a root among its 
descendants; but preorder and postorder carry over in an obvious manner. Given 
any nonempty forest, the two basic ways to traverse it may be defined as follows: 


Preorder traversal Postorder traversal 
Visit the root of the first tree Traverse the subtrees of the first tree 
Traverse the subtrees of the first tree Visit the root of the first tree 
Traverse the remaining trees Traverse the remaining trees 


In order to understand the significance of these two methods of traversal, 
consider the following notation for expressing tree structure by nested parenthe- 
ses: 

(A(B, C(K)), D(E(H), F(J), G)). (5) 
This notation corresponds to the forest (1): We represent a tree by the informa- 
tion written in its root, followed by a representation of its subtrees; we represent 
a nonempty forest by a parenthesized list of the representations of its trees, 
separated by commas. 

If (2) is traversed in preorder, we visit the nodes in the sequence A B CK D 
EHF JG; this is simply (5) with the parentheses and commas removed. Preorder 
is a natural way to list the nodes of a tree: We list the root first, then the 
descendants. If a tree structure is represented by indentation as in Fig. 20(c), 
the rows appear in preorder. The section numbers of this book itself (see Fig. 22) 
appear in preorder; thus, for example, Section 2.3 is followed by Section 2.3.1, 
then come Sections 2.3.2, 2.3.3, 2.3.4, 2.3.4.1, ..., 2.3.4.6, 2.3.5, 2.4, etc. 

It is interesting to note that preorder is a time-honored concept that might 
meaningfully be called dynastic order. At the death of a king, duke, or earl, the 
title passes to the first son, then to descendants of the first son, and finally if 
these all die out it passes to other sons of the family in the same way. (English 
custom also includes daughters in a family on the same basis as sons, except 
that they come after all the sons.) In theory, we could take a lineal chart of all 
the aristocracy and write out the nodes in preorder; then if we consider only the 
people presently living, we would obtain the order of succession to the throne 
(except as modified by Acts of Abdication). 

Postorder for the nodes in (1) is BK C A H E J F G D; this is analogous 
to preorder, except that it corresponds to the similar parenthesis notation 


((B, (K)C)A, ((H)E, (J)F, G)D), (6) 


in which a node appears just after its descendants instead of just before. 

The definitions of preorder and postorder mesh very nicely with the natural 
correspondence between trees and binary trees, since the subtrees of the first 
tree correspond to the left binary subtree, and the remaining trees correspond to 
the right binary subtree. By comparing these definitions with the corresponding 
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definitions on page 319, we find that traversing a forest in preorder is exactly 
the same as traversing the corresponding binary tree in preorder. Traversing a 
forest in postorder is exactly the same as traversing the corresponding binary 
tree in inorder. The algorithms developed in Section 2.3.1 may therefore be 
used without change. (Note that postorder for trees corresponds to inorder, 
not postorder, for binary trees. This is fortunate, since we have seen that it 
is comparatively hard to traverse binary trees in postorder.) Because of this 
equivalence, we use the notation P$ for the postorder successor of node P ina 
tree or forest, while it denotes the inorder successor in a binary tree. 


As an example of the application of these methods to a practical problem, 
we will consider the manipulation of algebraic formulas. Such formulas are 
most properly regarded as representations of tree structures, not as one- or two- 
dimensional configurations of symbols, nor even as binary trees. For example, 
the formula y = 3ln(x + 1) — a/z? has the tree representation 


Here the illustration on the left is a conventional tree diagram like Fig. 21, in 
which the binary operators +, —, x, /, and f (the latter denotes exponentiation) 
have two subtrees corresponding to their operands; the unary operator “ln” has 
one subtree; variables and constants are terminal nodes. The illustration on the 
right shows the equivalent right-threaded binary tree, including an additional 
node y that is a list head for the tree. The list head has the form described in 
2.3.1-(8). 

It is important to note that, even though the left-hand tree in (7) bears 
a superficial resemblance to a binary tree, we are treating it here as a tree, 
and representing it by a quite different binary tree, shown at the right in (7). 
Although we could develop routines for algebraic manipulations based directly 
on binary tree structures — the so-called “three-address code” representations of 
algebraic formulas — several simplifications occur in practice if we use the general 
tree representation of algebraic formulas, as in (7), because postorder traversal 
is easier in a tree. 
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The nodes of the left-hand tree in (7) are 
- x 3 In + @ 1 / at «aw 2 in preorder; (8) 
3 x 1 + h x aae2t / = in postorder. (9) 


Algebraic expressions like (8) and (9) are very important, and they are known 
as “Polish notations” because form (8) was introduced by the Polish logician, 
Jan Łukasiewicz. Expression (8) is the prefix notation for formula (7), and (9) 
is the corresponding postfix notation. We will return to the interesting topic of 
Polish notation in later chapters; for now let us be content with the knowledge 
that Polish notation is directly related to the basic orders of tree traversal. 

We shall assume that tree structures for the algebraic formulas with which 
we will be dealing have nodes of the following form in MIX programs: 


RTAG| RLINK |TYPE| LLINK 
"INFO ) 


(10) 


Here RLINK and LLINK have the usual significance, and RTAG is negative for thread 
links (corresponding to RTAG = 1 in the statements of algorithms). The TYPE 
field is used to distinguish different kinds of nodes: TYPE = 0 means that the node 
represents a constant, and INFO is the value of the constant. TYPE = 1 means 
that the node represents a variable, and INFO is the five-letter alphabetic name 
of this variable. TYPE > 2 means that the node represents an operator; INFO is 
the alphabetic name of the operator and the value TYPE = 2, 3, 4, ... is used to 
distinguish the different operators +, —, x, /, etc. We will not concern ourselves 
here with how the tree structure has been set up inside the computer memory 
in the first place, since this topic is analyzed in great detail in Chapter 10; let us 
merely assume that the tree already appears in our computer memory, deferring 
questions of input and output until later. 

We shall now discuss the classical example of algebraic manipulation, finding 
the derivative of a formula with respect to the variable x. Programs for algebraic 
differentiation were among the first symbol-manipulation routines ever written 
for computers; they were used as early as 1952. The process of differentiation il- 
lustrates many of the techniques of algebraic manipulation, and it is of significant 
practical value in scientific applications. 

Readers who are not familiar with mathematical calculus may consider this 
problem as an abstract exercise in formula manipulation, defined by the following 
rules: 
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D(u/v) = D(u)/v- (u x D(v)) /(v + 2) (18) 

D(u fv) = D(u) x (v x (ut (v — 1))) + (Inu) x D(v)) x (ut v) (19) 
These rules allow us to evaluate the derivative D(y) for any formula y composed 
of the operators listed. The “—” sign in rule (14) is a unary operator, which 
is different from the binary “—” in (16); we will use “neg” to stand for unary 
negation in the tree nodes below. 

Unfortunately rules (11)—(19) don’t tell the whole story. If we apply them 
blindly to a rather simple formula like 


y = 3ln(z + 1) — a/2’, 


we get 


D(y) =0-In(x +1) +3((1+0)/(z + 1)) 
— (0/2? — (a(1 (2z?) + ((Inz)- Oe yay), (20) 


which is correct but totally unsatisfactory. To avoid so many redundant opera- 
tions in the answer, we must recognize the special cases of adding or multiplying 
by zero, multiplying by one, or raising to the first power. These simplifications 
reduce (20) to 

D(y) = 3(1/( + 1) — (—(a(22))/(2*)’), (21) 
which is more acceptable but still not ideal. The concept of a really satisfactory 
answer is not well-defined, because different mathematicians will prefer formulas 
to be expressed in different ways; however, it is clear that (21) is not as simple 
as it could be. In order to make substantial progress over formula (21), it is 
necessary to develop algebraic simplification routines (see exercise 17), which 
would reduce (21) to, for example, 


D(y) = 3(x + 1) + 2ax-*. (22) 


We will content ourselves here with routines that can produce (21), not (22). 

Our main interest in this algorithm is, as usual, in the details of how the 
process is carried out inside a computer. Many higher-level languages and special 
routines are available at most computer installations, with built-in facilities 
to simplify algebraic manipulations like these; but the purpose of the present 
example is to gain more experience in fundamental tree operations. 

The idea behind the following algorithm is to traverse the tree in postorder, 
forming the derivative of each node as we go, until eventually the entire derivative 
has been calculated. Using postorder means that we will arrive at an operator 
node (like “+”) after its operands have been differentiated. Rules (11) through 
(19) imply that every subformula of the original formula will have to be differen- 
tiated, sooner or later, so we might as well do the differentiations in postorder. 

By using a right-threaded tree, we avoid the need for a stack during the 
operation of the algorithm. On the other hand, a threaded tree representation 
has the disadvantage that we will need to make copies of subtrees; for example, 
in the rule for D(u t v) we might need to copy u and v three times each. If we 
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had chosen to use a List representation as in Section 2.3.5 instead of a tree, we 
could have avoided such copying. 


Algorithm D (Differentiation). If Y is the address of a list head that points to 
a formula represented as described above, and if DY is the address of the list head 
for an empty tree, this algorithm makes NODE(DY) point to a tree representing 
the analytic derivative of Y with respect to the variable “X”. 


D1. [Initialize.] Set P + Ys (namely, the first node of the tree, in postorder, 
which is the first node of the corresponding binary tree in inorder). 

D2. [Differentiate.] Set P1 4 LLINK(P); and if P1 # A, also set Q1 <- RLINK (P1). 
Then perform the routine DIFF [TYPE(P)], described below. (The routines 
DIFF[0], DIFF[1], etc., will form the derivative of the tree with root P, and 
will set pointer variable Q to the address of the root of the derivative. The 
variables P1 and Q1 are set up first, in order to simplify the specification of 
the DIFF routines.) 


D3. [Restore link.] If TYPE(P) denotes a binary operator, set RLINK(P1) + P2. 
(See the next step for an explanation.) 

D4. [Advance to Ps.] Set P2 «+ P, P + Ps. Now if RTAG(P2) = 0 (that is, if 
NODE(P2) has a sibling to the right), set RLINK(P2) <+ Q. (This is the tricky 
part of the algorithm: We temporarily destroy the structure of tree Y, so 
that a link to the derivative of P2 is saved for future use. The missing link 
will be restored later in step D3. See exercise 21 for further discussion of 
this trick.) 

D5. [Done?] If P Æ Y, return to step D2. Otherwise set LLINK(DY) + Q and 
RLINK(Q) + DY, RTAG(Q) = 1. J 
The procedure described in Algorithm D is just the background routine 

for the differentiation operations that are performed by the processing routines 

DIFF [0], DIFF[1], ..., called in step D2. In many ways, Algorithm D is like the 

control routine for an interpretive system or machine simulator, as discussed in 

Section 1.4.3, but it traverses a tree instead of a simple sequence of instructions. 
To complete Algorithm D we must define the routines that do the actual 

differentiation. In the following discussion, the statement “P points to a tree” 

means that NODE(P) is the root of a tree stored as a right-threaded binary tree, 
although both RLINK(P) and RTAG(P) will be meaningless so far as this tree is 
concerned. We will make use of a tree construction function that makes new 

trees by joining smaller ones together: Let x denote some kind of node, either a 

constant, variable, or operator, and let U and V denote pointers to trees; then 
TREE(z,U,V) makes a new tree with x in its root node and with U and V 

the subtrees of the root: W <= AVAIL, INFO(W) «+ x, LLINK(W) + U, 
RLINK(U) + V, RTAG(U) + 0, RLINK(V) + W, RTAG(V) < 1. 
TREE(2,U) similarly makes a new tree with only one subtree: W <= AVAIL, 
INFO(W) + z, LLINK(W) < U, RLINK(U) + W, RTAG(U) < 1. 
TREE(a2) makes a new tree with x as a terminal root node: W <= AVAIL, 
INFO(W) < x, LLINK(W) + A. 
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Furthermore TYPE(W) is set appropriately, depending on x. In all cases, the 
value of TREE is W, that is, a pointer to the tree just constructed. The reader 
should study these three definitions carefully, since they illustrate the binary 
tree representation of trees. Another function, COPY(U), makes a copy of the 
tree pointed to by U and has as its value a pointer to the tree thereby created. 
The basic functions TREE and COPY make it easy to build up a tree for the 
derivative of a formula, step by step. 


Nullary operators (constants and variables). For these operations, NODE(P) 
is a terminal node, and the values of P1, P2, Q1, and Q before the operation are 
irrelevant. 


DIFF [0]: (NODE(P) is a constant.) Set Q << TREE (0). 
DIFF[1]: (NODE(P) is a variable.) If INFO(P) = “X”, set Q + TREE(1); 
otherwise set Q < TREE(0). 


Unary operators (logarithm and negation). For these operations, NODE(P) has 
one child, U, pointed to by P1, and Q points to D(U). The values of P2 and Q1 
before the operation are irrelevant. 


DIFF [2]: (NODE(P) is “In”.) If INFO(Q) 40, set Q4 TREE(“/”,Q,COPY(P1)). 
DIFF [3]: (NODE(P) is “neg”.) If INFO(Q) 4 0, set Q + TREE(“neg”,Q). 
Binary operators (addition, subtraction, multiplication, division, exponentia- 
tion). For these operations, NODE(P) has two children, U and V, pointed to 

respectively by P1 and P2; Q1 and Q point respectively to D(U), D(V). 
DIFF [4]: (“+” operation.) If INFO(Q1) = 0, set AVAIL = Q1. Otherwise if 
INFO(Q) = 0, set AVAIL <+ Q and Q + Q1; otherwise set Q + TREE(“+”,Q1,Q). 


DIFF [5]: (“—” operation.) If INFO(Q) = 0, set AVAIL <= Q and Q & Q1. 
Otherwise if INFO(Q1) = 0, set AVAIL < Q1 and set Q + TREE(“neg”,Q); 
otherwise set Q +— TREE(“—”,Q1,Q). 


DIFF [6]: (“x” operation.) If INFO(Q1) Æ 0, set Q1 + MULT(Q1,COPY(P2)). 
Then if INFO(Q) 4 0, set Q +— MULT(COPY(P1) ,Q). Then go to DIFF[4]. 


Here MULT(U,V) is a new function that constructs a tree for U x V but also 
makes a test to see if U or V is equal to 1: 


if INFO(U) = 1 and TYPE(U) = 0, set AVAIL < U and MULT(U,V) < V; 
if INFO(V) = 1 and TYPE(V) = 0, set AVAIL < V and MULT(U,V) + U; 
otherwise set MULT(U,V) + TREE(“x”,U,V). 


DIFF[7]: (“/” operation.) If INFO(Q1) Æ 0, set 
Q1 < TREE(“/”,Q1,COPY(P2)). 
Then if INFO(Q) 4 0, set 
Q < TREE(“/”,MULT(COPY(P1) ,Q) , TREE (“1”, COPY (P2) , TREE(2) )). 


Then go to DIFF[5]. 
DIFF(8]: (“t” operation.) See exercise 12. 
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We conclude this section by showing how all of the operations above are 
readily transformed into a computer program, starting “from scratch” with only 


MIX machine language as a basis. 


Program D (Differentiation). The following MIXAL program performs Algo- 
rithm D, with rl2 = P, rI3 = P2, rl4 = P1, rI5 = Q, rl6 = Q1. The order of 


computations has been rearranged a little, for convenience. 


001 * DIFFERENTIATION IN A RIGHT-THREADED TREE 

002 LLINK EQU 4:5 Definition of fields, see (10) 

003 RLINK EQU 1:2 

004 RLINKT EQU 0:2 

005 TYPE EQU 3:3 

006 * MAIN CONTROL ROUTINE D1. Initialize. 

007 D1 STJ 9F Treat the whole procedure as a subroutine. 
008 LD4 Y(LLINK) P1 + LLINK(Y), prepare to find Ys. 
009 1H ENT2 0,4 P + P1. 

010 2H LD4 0,2(LLINK) P1 <- LLINK(P). 

011 J4NZ 1B If P1 Æ A, repeat. 

012 D2 LD1 0,2(TYPE) D2. Differentiate. 

013 JMP *+1,1 Jump to DIFF [TYPE (P)]. 

014 JMP CONSTANT Switch to table entry for DIFF [0]. 
015 JMP VARIABLE DIFF[1]. 
016 JMP LN DIFF [2]. 
017 JMP NEG DIFF [3]. 
018 JMP ADD DIFF [4]. 
019 JMP SUB DIFF [5]. 
020 JMP MUL DIFF [6]. 
021 JMP DIV DIFF [7]. 
022 JMP PWR DIFF [8]. 
023 D3 ST3 0,4(RLINK) D3. Restore link. RLINK (P1) + P2. 
024 D4 ENT3 0,2 D4. Advance to Ps. P2 +P. 

025 LD2 0,2(RLINKT) P < RLINKT(P). 

026 J2N 1F Jump if RTAG(P) = 1; 

027 ST5 0,3(RLINK) otherwise set RLINK (P2) + Q. 

028 JMP 2B Note that NODE (P$) will be terminal. 
029 1H ENN2 0,2 

080 D5 ENT1 -Y,2 D5. Done? 

031 LD4 0,2(LLINK) P41 <+ LLINK(P), prepare for step D2. 
032 LD6 0O,4(RLINK) Q1 + RLINK(P1). 

033 J1NZ D2 Jump to D2 if P Æ Y; 

034 ST5 DY(LLINK) otherwise set LLINK (DY) + Q. 

035 ENNA DY 

036 STA 0,5(RLINKT) RLINK(Q) + DY, RTAG(Q) + 1. 

097 9H JMP * Exit from differentiation subroutine. J] 


The next part of the program contains the basic subroutines TREE and COPY. The 
former has three entrances TREEO, TREE1, and TREE2, according to the number 
of subtrees of the tree being constructed. Regardless of which entrance to the 
subroutine is used, rA will contain the address of a special constant indicating 
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what type of node forms the root of the tree being constructed; these special 
constants appear in lines 105-124. 


038 * BASIC SUBROUTINES FOR TREE CONSTRUCTION 
039 TREEO STJ 9F TREE (rA) function: 
040 JMP 2F 
041 TREE1 ST1 3F(0:2) TREE (rA ,rI1) function: 
042 JSJ 1F 
043 TREE2 STX 3F(0:2) TREE(rA,rX,rI1) function: 
044 3H ST1 *(RLINKT) RLINK(rX) ¢ rll, RTAG(rX) + 0. 
045 1H STJ 9F 
046 LDXN AVAIL 
O47 JXZ OVERFLOW 
048 STX 0,1(RLINKT) RLINK(rI1) + AVAIL, RTAG(rI1) + 1. 
049 LDX 3B(0:2) 
050 STA ¥*+1(0:2) 
051 STX *(LLINK) Set LLINK of next root node. 
052 2H LD1 AVAIL rll < AVAIL. 
053 J1Z OVERFLOW 
054 LDX 0,1(LLINK) 
055 STX AVAIL 
056 STA *+1(0:2) Copy root info to new node. 
057 MOVE *(2) 
058 DEC1 2 Reset rI1 to point to the new root. 
059 9H JMP * Exit from TREE, rll points to new tree. 
060 COPYP1 ENT1 0,4 COPY (P1), special entrance to COPY 
061 JSJ COPY 
062 COPYP2 ENT1 0,3 COPY (P2), special entrance to COPY 
063 COPY STJ 9F COPY (rI1) function: 
; see exercise 13) 
104 9H JMP * Exit from COPY, rll points to new tree. 
105 CONO CON O Node representing the constant “0” 
106 CON O 
107 CON1 CON O Node representing “1” 
108 CON 1 
109 CON2 CON O Node representing “2” 
110 CON 2 
111 LOG CON 2(TYPE) Node representing “In” 
112 ALF LN 
113 NEGOP CON 3(TYPE) Node representing “neg” 
114 ALF NEG 
115 PLUS CON 4(TYPE) Node representing “+” 
116 ALF + 
117 MINUS CON 5(TYPE) Node representing “—” 
118 ALF = 
119 TIMES CON 6(TYPE) Node representing “x” 
120 ALF * 
121 SLASH CON 7(TYPE) Node representing “/” 
122 ALF / 
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124 
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UPARROW 


CON 8(TYPE) 
ALF 


xk 


2.3.2 


Node representing “+” 


The remaining portion of the program corresponds to the differentiation routines 
DIFF [0], DIFF[1], ...; these routines are written to return control to step D3 
after processing a binary operator, otherwise they return to step D4. 


125 
126 
127 
128 
129 
130 
131 
132 
133 
134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
151 
152 
153 
154 
155 
156 
157 
158 
159 
160 
161 
162 
163 
164 
165 
166 
167 
168 


* DIFFERENTIATION ROUTINES 


VARIABLE 


CONSTANT 


1H 


2H 
LN 


NEG 


ADD 


3H 


1H 


2H 


1H 
4H 


SUB 


LDX 
ENTA 
CMPX 
JE 
ENTA 
JMP 
ENTS 
JMP 
ALF 
LDA 


a 
> 
N 


SURMRMorouAms 


SS 42220 


WUVHHNPFPVIUHY 


w2 
CON1 
2F 
*+2 
CONO 
TREEO 
0,1 
D4 


COPYP1 
0,5 
SLASH 
TREE2 
1B 

1,5 
D4 
NEGOP 
0,5 
TREE1 
1B 

1,6 

iF 
AVAIL 
0,6(LLINK) 
AVAIL 
D3 

1,5 

iF 
AVAIL 
0,5(LLINK) 
AVAIL 


Is INFO(P) = “X”? 
If so, call TREE(1). 
Call TREE (0). 


Q < location of new tree. 
Return to control routine. 


Return to control routine if INFO(Q) = 0; 
otherwise set rll + COPY(P1). 


rll + TREE(“/”,Q,rI1). 
Q < rll, return to control. 


Return if INFO(Q) = 0. 


rll + TREE(“neg” ,Q). 
Q < rll, return to control. 


Jump unless INFO(Q1) = 0. 
AVAIL © Q1. 


Return to control, binary operator. 
Jump unless INFO(Q) = 0. 


AVAIL = Q. 


Q<¢ Ql. 
Return to control. 
Prepare to call TREE(“+” ,Q1,Q). 


Q «+ TREE(“+”,Q1,Q). 
Return to control. 


Jump if INFO(Q) = 0. 
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169 
170 
171 

172 
173 
114 

175 
176 
177 
178 
179 
180 
181 

182 
183 
184 

185 
186 
187 
188 
189 
190 
191 

192 
193 
194 

195 
196 
197 
198 
199 
200 
201 

202 
203 
204 

205 
206 
207 
208 
209 
210 
211 

212 
213 
214 
215 
216 
217 


1H 


MUL 


1H 


MULT 


1H 


1H 


2H 


1H 


8H 
9H 


LDA 


a 
> 
a 
N 


NTA 


s 24 


2 zZ 


= 


Z2SeErvuaeases 235 rPug 
YQUMWHA VHA HUN EPH VHA UN PVH VHAVUSHY 
o1 e. e a > > o = 


222 


QMUMSHROOPHSGMSOPoRonSe BS 


UN WN 
AHA 
Ne 


ENT2 
LDA 
DECA 
JANZ 
LDA 
JAZ 
LDA 
DECA 
JANZ 
LDA 
JANZ 
ST1 
ENT1 
ENT2 
LDA 
STA 
ST2 
JMP 
ENTA 
ENTX 
JMP 
ENT2 
JMP 
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1,6 

1F 
NEGOP 
0,5 
TREE1 
0,1 

3B 
MINUS 
4B 

1,6 

1F 
COPYP2 
0,6 
MULT 
0,1 
1,5 
ADD 
COPYP1 
0,1 
0,5 
MULT 
0,1 
ADD 

9F 
1F(0:2) 
8F(0:2) 
* 

1,2 

1 

1F 
0,2(TYPE) 
2F 

1,1 

1 

1F 
0,1(TYPE) 
1F 

*+2 (0:2) 
0,2 

* 

AVAIL 
0,2(LLINK) 
AVAIL 
8F 
TIMES 
0,2 
TREE2 
* 

* 


Jump unless INFO(Q1) = 0. 


Q «+ TREE(“neg” ,Q). 
AVAIL < Q1 and return. 
Prepare to call TREE(“—” ,Q1,Q). 


Jump if INFO(Q1) = 0; 
otherwise set rll < COPY (P2). 


rll + MULT(Q1,COPY(P2) ). 
Qi< rll. 


Jump if INFO(Q) = 0; 
otherwise set rll < COPY(P1). 
rll < MULT(COPY(P1) ,Q). 
Q< rill. 
MULT (rA ,rI1) subroutine: 
Let rA = U, rll =V. 
Save rI2. 
rl2 + U. 
Test if INFO(U) = 1 


and if TYPE(U) = 0. 


If not, test if INFO(V) = 1 


and if TYPE(V) = 0. 


If so, interchange U 4 V. 


AVAIL <& U. 


Result is V. 


Result is TREE(“x”,U,V). 
Restore rI2 setting. 
Exit MULT with result in rll. J 
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The other two routines DIV and PWR are similar and they have been left as 
exercises (see exercises 15 and 16). 


EXERCISES 


1. [20] The text gives a formal definition of B(F), the binary tree corresponding to 
a forest F. Give a formal definition that reverses the process; in other words, define 
F(B), the forest corresponding to a binary tree B. 


2. [20] We defined Dewey decimal notation for forests in Section 2.3, and for binary 
trees in exercise 2.3.1-5. Thus the node “J” in (1) is represented by “2.2.1”, and in 
the equivalent binary tree (3) it is represented by “11010”. If possible, give a rule 
that directly expresses the natural correspondence between trees and binary trees as a 
correspondence between the Dewey decimal notations. 


3. [22] What is the relation between Dewey decimal notation for the nodes of a 
forest and the preorder and postorder of those nodes? 


4. [19] Is the following statement true or false? “The terminal nodes of a tree occur 
in the same relative position in preorder and postorder.” 


5. [23] Another correspondence between forests and binary trees could be defined by 
letting RLINK(P) point to the rightmost child of NODE(P), and LLINK(P) to the nearest 
sibling on the left. Let F be a forest that corresponds in this way to a binary tree B. 
What order, on the nodes of B, corresponds to (a) preorder (b) postorder on F? 


6. [25] Let T be a nonempty binary tree in which each node has 0 or 2 children. If 
we regard T as an ordinary tree, it corresponds (via the natural correspondence) to 
another binary tree T”. Is there any simple relation between preorder, inorder, and 
postorder of the nodes of T (as defined for binary trees) and the same three orders for 
the nodes of T’? 


7. [M20] A forest may be regarded as a partial ordering, if we say that each node 
precedes its descendants in the tree. Are the nodes topologically sorted (as defined in 
Section 2.2.3) when they are listed in (a) preorder? (b) postorder? (c) reverse preorder? 
(d) reverse postorder? 


8. [M20] Exercise 2.3.1-25 shows how an ordering between the information stored 
in the individual nodes of a binary tree may be extended to a linear ordering of all 
binary trees. The same construction leads to an ordering of all trees, under the natural 
correspondence. Reformulate the definition of that exercise, in terms of trees. 


9. [M21] Show that the total number of nonterminal nodes in a forest has a simple 
relation to the total number of right links equal to A in the corresponding unthreaded 
binary tree. 


10. [M23] Let F be a forest of trees whose nodes in preorder are wi, U2,...,Un, and 
let F’ be a forest whose nodes in preorder are uj, ub,..., uh.. Let d(u) denote the 
degree (the number of children) of node u. In terms of these ideas, formulate and prove 
a theorem analogous to Theorem 2.3.1A. 


11. [15] Draw trees analogous to those shown in (7), corresponding to the formula 
as 

12. [M21] Give specifications for the routine DIFF[8] (the “t” operation), which was 
omitted from the algorithm in the text. 
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> 13. [26] Write a MIX program for the COPY subroutine (which fits in the program of 
the text between lines 063-104). [Hint: Adapt Algorithm 2.3.1C to the case of right- 
threaded binary trees, with suitable initial conditions.] 


> 14. [M21] How long does it take the program of exercise 13 to copy a tree with n 
nodes? 


15. [23] Write a MIX program for the DIV routine, corresponding to DIFF[7] as speci- 
fied in the text. (This routine should be added to the program in the text after line 217.) 


16. [24] Write a MIX program for the PWR routine, corresponding to DIFF [8] as speci- 
fied in exercise 12. (This routine should be added to the program in the text after the 
solution to exercise 15.) 


17. [M40] Write a program to do algebraic simplification capable of reducing, for 
example, (20) or (21) to (22). [Hints: Include a new field with each node, representing 
its coefficient (for summands) or its exponent (for factors in a product). Apply algebraic 
identities, like replacing In(u f v) by vlnu; remove the operations —, /, t, and neg 
when possible by using equivalent addition or multiplication operations. Make + and 
x into n-ary instead of binary operators; collect like terms by sorting their operands 
in tree order (exercise 8); some sums and products will now reduce to zero or unity, 
presenting perhaps further simplifications. Other adjustments, like replacing a sum of 
logarithms by the logarithm of a product, also suggest themselves. | 


> 18. [25] An oriented tree specified by n links PARENT[j] for 1 < j < n implicitly defines 
an ordered tree if the nodes in each family are ordered by their location. Design an 
efficient algorithm that constructs a doubly linked circular list containing the nodes of 
this ordered tree in preorder. For example, given 


j=12345678 
PARENT[|j] = 38404834 


your algorithm should produce 


LLINK[j] =3 8467215 
RLINK[|j] =76138452 


and it should also report that the root node is 4. 


19. [M35] A free lattice is a mathematical system, which (for the purposes of this 
exercise) can be simply defined as the set of all formulas composed of variables and 
two abstract binary operators “V” and “A”. A relation “X > Y” is defined between 
certain formulas X and Y in the free lattice by the following rules: 


i) XVY > WAZ if and only if X VY > Wor XVY >= Zor X =>WAZ or 
Y>WAZ; 


ii) X AY > Z if and only if X > Z and Y > Z; 
X >Y VZ if and only if X > Y and X > Z; 
x> Y AZ if and only if x > Y or x > Z, when z is a variable; 
X VY > z if and only if X > z or Y > z, when z is a variable; 


) 
iii) 
iv) 
v) 
vi) x > y if and only if x = y, when z and y are variables. 


For example, we find a A (bV c) = (a ^b) V (adc) Za (bVc). 
Design an algorithm that tests whether or not X > Y, given two formulas X and Y 
in the free lattice. 
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> 20. [M22] Prove that if u and v are nodes of a forest, u is a proper ancestor of v if 
and only if u precedes v in preorder and u follows v in postorder. 


21. [25] Algorithm D controls the differentiation activity for binary operators, unary 
operators, and nullary operators, thus for trees whose nodes have degree 2, 1, and 0; but 
it does not indicate explicitly how the control would be handled for ternary operators 
and nodes of higher degree. (For example, exercise 17 suggests making addition and 
multiplication into operators with any number of operands.) Is it possible to extend 
Algorithm D in a simple way so that it will handle operators of degree more than 2? 


> 22. [M26] If T and T” are trees, let us say T can be embedded in T’, written T C T’, 
if there is a one-to-one function f from the nodes of T into the nodes of T’ such that 
f preserves both preorder and postorder. (In other words, u precedes v in preorder 
for T if and only if f(u) precedes f(v) in preorder for T’, and the same holds for 
postorder. See Fig. 25.) 


Fig. 25. One tree embedded in another (see exercise 22). 


If T has more than one node, let I(T) be the leftmost subtree of root(T) and let 
r(T) be the rest of T, that is, T with I(T) deleted. Prove that T can be embedded in 
T” if (i) T has just one node, or (ii) both T and T’ have more than one node and either 
T CUT’), or T C r(T'), or (I(T) C I(T’) and r(T) C r(T’)). Does the converse hold? 


2.3.3. Other Representations of Trees 


There are many ways to represent tree structures inside a computer besides the 
LLINK-RLINK (left child—right sibling) method given in the previous section. 
As usual, the proper choice of representation depends heavily on what kind of 
operations we want to perform on the trees. In this section we will consider a 
few of the tree representation methods that have proved to be especially useful. 

First we can use sequential memory techniques. As in the case of linear lists, 
this mode of allocation is most suitable when we want a compact representation 
of a tree structure that is not going to be subject to radical dynamic changes in 
size or shape during program execution. There are many situations in which we 
need essentially constant tables of tree structures for reference within a program, 
and the desired form of these trees in memory depends on the way in which the 
tables are to be examined. 

The most common sequential representation of trees (and forests) corres- 
ponds essentially to the omission of LLINK fields, by using consecutive addressing 
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instead. For example, let us look again at the forest 
(A(B, C(K)), D(E(H), F(J),G)) (1) 
considered in the previous section, which has the tree diagrams 
(A) Z and © = 
© © © © @—-© E 


© ® © O @ (2) 


The preorder sequential representation has the nodes appearing in preorder, 
with the fields INFO, RLINK, and LTAG in each node: 


RLINK ~y aas | (3) 
INFO ABCK DEH F JG 3 
LTAG had Jot 


Here nonnull RLINKs have been indicated by arrows, and LTAG = 1 (for terminal 
nodes) is indicated by “|”. LLINK is unnecessary, since it would either be null 
or it would point to the next item in sequence. It is instructive to compare (1) 
with (3). 

This representation has several interesting properties. In the first place, 
all subtrees of a node appear immediately after that node, so that all subtrees 
within the original forest appear in consecutive blocks. [Compare this with the 
“nested parentheses” in (1) and in Fig. 20(b).] In the second place, notice that 
the RLINK arrows never cross each other in (3); this will be true in general, for 
in a binary tree all nodes between X and RLINK(X) in preorder lie in the left 
subtree of X, hence no outward arrows will emerge from that part of the tree. 
In the third place, we may observe that the LTAG field, which indicates whether 
a node is terminal or not, is redundant, since “|” occurs only at the end of the 
forest and just preceding every downward pointing arrow. 

Indeed, these remarks show that the RLINK field itself is almost redundant; 
all we really need to represent the structure is RTAG and LTAG. Thus it is possible 
to deduce (3) from much less data: 


RTAG | | 
INFO ABCK DEH PF J.¢G (4) 
LTAG | a 


As we scan (4) from left to right, the positions with RTAG 4 “|” correspond 
to nonnull RLINKs that must be filled in. Each time we pass an item with 
LTAG = “|”, we should complete the most recent instance of an incomplete 
RLINK. (The locations of incomplete RLINKs can therefore be kept on a stack.) 
We have essentially proved Theorem 2.3.1A again. 

The fact that RLINK or LTAG is redundant in (3) is of little or no help to 
us unless we are scanning the entire forest sequentially, since extra computation 
is required to deduce the missing information. Therefore we often need all of 
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the data in (3). However, there is evidently some wasted space, since more than 
half of the RLINK fields are equal to A for this particular forest. There are two 
common ways to make use of the wasted space: 

1) Fill the RLINK of each node with the address following the subtree below 
that node. The field is now often called “SCOPE” instead of RLINK, since it 
indicates the right boundary of the “influence” (descendants) of each node. 
Instead of (3), we would have 


SCOPE 
TAT are (5) 
INFO ABCKDEHFJG 
The arrows still do not cross each other. Furthermore, LTAG(X) = “|” is char- 


acterized by the condition SCOPE(X) = X+c, where c is the number of words per 
node. One example of the use of this SCOPE idea appears in exercise 2.4—12. 

2) Decrease the size of each node by removing the RLINK field, and add 
special “link” nodes just before nodes that formerly had a nonnull RLINK: 


INFO DA oe ee a (6) 
x x x x 
LTAG | | E i 


(CEOE 


Here “x” indicates the special link nodes, whose INFO somehow characterizes 
them as links pointing as shown by the arrows. If the INFO and RLINK fields of 
(3) occupy roughly the same amount of space, the net effect of the change to (6) 
is to consume less memory, since the number of “*” nodes is always less than 
the number of non-“*«” nodes. Representation (6) is somewhat analogous to a 
sequence of instructions in a one-address computer like MIX, with the “*” nodes 
corresponding to conditional jump instructions. 

Another sequential representation analogous to (3) may be devised by omit- 
ting RLINKs instead of LLINKs. In this case we list the nodes of the forest in a new 
order that may be called family order since the members of each family appear 
together. Family order for any forest may be defined recursively as follows: 


Visit the root of the first tree. 

Traverse the remaining trees (in family order). 

Traverse the subtrees of the root of the first tree (in family order). 
(Compare this with the definitions of preorder and postorder in the previous sec- 
tion. Family order is identical with the reverse of postorder in the corresponding 
binary tree.) 

The family order sequential representation of the trees (2) is 


LLINK | CHUD O y } | ry 
INFO ADEFG JHBCK (7) 
RTAG l PJs A 
In this case the RTAG entries serve to delimit the families. Family order begins 
by listing the roots of all trees in the forest, then continues by listing individual 
families, successively choosing the family of the most recently appearing node 
whose family has not yet been listed. It follows that the LLINK arrows will 
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never cross; and the other properties of preorder representation carry over in a 
similar way. 

Instead of using family order, we could also simply list the nodes from left 
to right, one level at a time. This is called “level order” [see G. Salton, CACM 5 
(1962), 103-114], and the level order sequential representation of (2) is 


LLINK a a ae 
INFO ADBCEFGKHJ (8) 
RTAG Jo ed 


This is like (7), but the families are chosen in first-in-first-out fashion rather than 
last-in-first-out. Either (7) or (8) may be regarded as a natural analog, for trees, 
of the sequential representation of linear lists. 

The reader will easily see how to design algorithms that traverse and analyze 
trees represented sequentially as above, since the LLINK and RLINK information 
is essentially available just as though we had a fully linked tree structure. 

Another sequential method, called postorder with degrees, is somewhat dif- 
ferent from the techniques above. We list the nodes in postorder and give the 
degree of each node instead of links: 


DEGREE 00120104103 (9) 
INFO BKCAHEJFGOD 9 


For a proof that this is sufficient to characterize the tree structure, see exercise 
2.3.2-10. This order is useful for the “bottom-up” evaluation of functions defined 
on the nodes of a tree, as in the following algorithm. 


Algorithm F (Evaluate a locally defined function in a tree). Suppose f is a 
function of the nodes of a tree, such that the value of f at a node x depends only 
on x and the values of f on the children of x. The following algorithm, using an 
auxiliary stack, evaluates f at each node of a nonempty forest. 

F1. [Initialize.] Set the stack empty, and let P point to the first node of the forest 
in postorder. 

F2. [Evaluate f.] Set d < DEGREE(P). (The first time this step is reached, d will 
be zero. In general, when we get to this point, it will always be true that 
the top d items of the stack are f(xa), ..., f(a1)— from the top of the 
stack downward—where z1, ..., xq are the children of NODE(P) from left 
to right.) Evaluate f(NODE(P)), using the values of f(xq), ..., f(a1) found 
on the stack. 

F3. [Update the stack.] Remove the top d items of the stack; then put the value 
f (NODE(P)) on top of the stack. 

F4. [Advance.] If P is the last node in postorder, terminate the algorithm. 
(The stack will then contain f(root(Tm)), ..., f(root(T;)), from top to 
bottom, where T, ..., Tm are the trees of the given forest.) Otherwise 
set P to its successor in postorder (this would be simply P + P + c in the 
representation (9), and return to step F2. I 
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The validity of Algorithm F follows by induction on the size of the trees 
processed (see exercise 16). This algorithm bears a striking similarity to the 
differentiation procedure of the previous section (Algorithm 2.3.2D), which eval- 
uates a function of a closely related type; see exercise 3. The same idea is used 
in many interpretive routines in connection with the evaluation of arithmetic 
expressions in postfix notation; we will return to this topic in Chapter 8. See 
also exercise 17, which gives another important procedure similar to Algorithm F. 


Thus we have seen various sequential representations of trees and forests. 
There are also a number of linked forms of representation, which we shall now 
consider. 

The first idea is related to the transformation that takes (3) into (6): We 
remove the INFO fields from all nonterminal nodes and put this information as 
a new terminal node below the previous node. For example, the trees (2) would 


become 
Q (*) 
4 © O o O O © (10) 


OIO DAOG 


This new form shows that we may assume (without loss of generality) that 
all INFO in a tree structure appears in its terminal nodes. Therefore in the 
natural binary tree representation of Section 2.3.2, the LLINK and INFO fields 
are mutually exclusive and they can share the same field in each node. A node 
might have the fields 


LTAG| LLINK or INFO RLINK 


where the sign LTAG tells whether the second field is a link or not. (Compare 
this representation with, for example, the two-word format of (10) in Section 
2.3.2.) By cutting INFO down from 5 bytes to 3, we can fit each node into one 
word. However, notice that there are now 15 nodes instead of 10; the forest (10) 
takes 15 words of memory while (2) takes 20, yet the latter has 50 bytes of INFO 
compared to 30 in the other. There is no real gain in memory space in (10) 
unless the excess INFO space was going to waste; the LLINKs replaced in (10) 
are removed at the expense of about the same number of new RLINKs in the 
added nodes. Precise details of the differences between the two representations 
are discussed in exercise 4. 

In the standard binary tree representation of a tree, the LLINK field might 
be more accurately called the LCHILD field, since it points from a parent node to 
its leftmost child. The leftmost child is usually the “youngest” of the children 
in the tree, since it is easier to insert a node at the left of a family than at the 
right; so the abbreviation LCHILD may also be thought of as the “last child” or 
“least child.” 
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Many applications of tree structures require rather frequent references up- 
ward in the tree as well as downward. A threaded tree gives us the ability to 
go upward, but not with great speed; we can sometimes do better if we have a 
third link, PARENT, in each node. This leads to a triply linked tree, where each 
node has LCHILD, RLINK, and PARENT links. Figure 26 shows a triply linked 
tree representation of (2). For an example of the use of triply linked trees, see 
Section 2.4. 


INFO |PARENT 
LCHILD| RLINK Fig. 26. A triply linked tree. 


D 
5 


B C ELS E Es 
+ o> e | @ $| o ele Se o 
A J 


It is clear that the PARENT link all by itself is enough to specify any oriented 
tree (or forest) completely. For we can draw the diagram of the tree if we know all 
the upward links. Every node except the root has just one parent, but there may 
be several children; so it is simpler to give upward links than downward ones. 
Why then haven’t we considered upward links much earlier in our discussion? 
The answer, of course, is that upward links by themselves are hardly adequate 
in most situations, since it is very difficult to tell quickly if a node is terminal 
or not, or to locate any of its children, etc. There is, however, a very important 
application in which upward links are sufficient by themselves: We now turn to 
a brief study of an elegant algorithm for dealing with equivalence relations, due 
to M. J. Fischer and B. A. Galler. 

An equivalence relation “=” is a relation between the elements of a set of 
objects S satisfying the following three properties for any objects x, y, and z 
(not necessarily distinct) in S: 

i) If x = y and y =z, then x = z. (Transitivity.) 

ii) If x = y, then y = x. (Symmetry.) 
iii) « = x. (Reflexivity.) 
(Compare this with the definition of a partial ordering relation in Section 2.2.3; 
equivalence relations are quite different from partial orderings, in spite of the fact 
that two of the three defining properties are the same.) Examples of equivalence 
relations are the relation “=”, the relation of congruence (modulo m) for integers, 
the relation of similarity between trees as defined in Section 2.3.1, etc. 
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The equivalence problem is to read in pairs of equivalent elements and to 
determine later whether two particular elements can be proved equivalent or 
not on the basis of the given pairs. For example, suppose that S is the set 
{1,2,3,4,5,6,7,8,9} and suppose that we are given the pairs 


1=5, 6=8, 7=2, 9=8, 3=7, 4=2, 9=3. (11) 


It follows that, for example, 2 = 6, since 2 = 7 = 3 = 9 = 8 = 6. But we cannot 
show that 1 = 6. In fact, the pairs (11) divide S into two classes 


{1,5} and  {2,3,4,6,7,8,9}, (12) 


such that two elements are equivalent if and only if they belong to the same 
class. It is not difficult to prove that any equivalence relation partitions its set S$ 
into disjoint classes (called the equivalence classes), such that two elements are 
equivalent if and only if they belong to the same class. 

Therefore a solution to the equivalence problem is a matter of keeping track 
of equivalence classes like (12). We may start with each element alone in its 
class, thus: 

{1} {2} {3} {4} {5} {6} {7} {8} {9} (13) 
Now if we are given the relation 1 = 5, we put {1,5} together in a class. After 
processing the first three relations 1 = 5, 6 = 8, and 7 = 2, we will have changed 
(13) to 
{1,5} {2,7} {3} {4} {6,8} {9}. (14) 
Now the pair 9 = 8 puts {6,8,9} together, etc. 

The problem is to find a good way to represent situations like (12), (13), 
and (14) within a computer so that we can efficiently perform the operations 
of merging classes together and of testing whether two given elements are in 
the same class. The algorithm below uses oriented tree structures for this 
purpose: The elements of S become nodes of an oriented forest; and two nodes 
are equivalent, as a consequence of the equivalent pairs read so far, if and only if 
they belong to the same tree. This test is easy to make, since two elements are in 
the same tree if and only if they are below the same root element. Furthermore, 
it is easy to merge two oriented trees together, by simply attaching one as a new 
subtree of the other’s root. 


Algorithm E (Process equivalence relations). Let S be the set of numbers 
{1,2,...,n}, and let PARENT[1], PARENT[2], ..., PARENT[n] be integer vari- 
ables. This algorithm inputs a set of relations such as (11) and adjusts the PARENT 
table to represent a set of oriented trees, so that two elements are equivalent as 
a consequence of the given relations if and only if they belong to the same 
tree. (Note: In a more general situation, the elements of S would be symbolic 
names instead of simply the numbers from 1 to n; then a search routine, as in 
Chapter 6, would locate nodes corresponding to the elements of S, and PARENT 
would be a field in each node. The modifications for this more general case are 
straightforward.) 
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E1. [Initialize.]| Set PARENT[k] «+ 0 forl<k<n. (T his means that all trees 
initially consist of a root alone, as in (13).) 

E2. [Input new pair.] Get the next pair of equivalent elements “j = k” from the 
input. If the input is exhausted, the algorithm terminates. 

E3. [Find roots.] If PARENT[j] > 0, set j + PARENT[j] and repeat this step. 
If PARENT[A] > 0, set k + PARENT[K] and repeat this step. (After this 
operation, 7 and k have moved up to the roots of two trees that are to be 
made equivalent. The input relation 7 = k was redundant if and only if we 
now have j = k.) 

E4. [Merge trees.] If j 4 k, set PARENT[j] + k. Go back to step E2. I 


The reader should try this algorithm on the input (11). After processing 
1=5,6=8, 7 = 2, and 9 = 8, we will have 


PARENT [K]: 5 0 0 0 
k : 123 4 


which represents the trees 


SOR. i 


After this point, the remaining relations of (11) are somewhat more interesting; 
see exercise 9. 

This equivalence problem arises in many applications. We will discuss 
significant refinements of Algorithm E in Section 7.4.1, when we study the 
connectivity of graphs. A more general version of the problem, which arises when 
a compiler processes “equivalence declarations” in languages like FORTRAN, is 
discussed in exercise 11. 


f 


Fig. 27. A ring structure. 


There are still more ways to represent trees in computer memory. Recall that 
we discussed three principal methods for representing linear lists in Section 2.2: 
the straight representation with terminal link A, the circularly linked lists, and 
the doubly linked lists. The representation of unthreaded binary trees described 
in Section 2.3.1 corresponds to a straight representation in both LLINKs and 
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Fig. 28. Representation of polynomials using four-directional links. Shaded areas of nodes indicate information that is irrelevant in ` 
the context considered. w 
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RLINKs. It is possible to get eight other binary tree representations by indepen- 
dently using any of these three methods in the LLINK and RLINK directions. For 
example, Fig. 27 shows what we get if circular linking is used in both directions. 
If circular links are used throughout as in the figure, we have what is called a 
ring structure; ring structures have proved to be quite flexible in a number of 
applications. The proper choice of representation depends, as always, on the 
types of insertions, deletions, and traversals that are needed in the algorithms 
that manipulate these structures. A reader who has looked over the examples 
given so far in this chapter should have no difficulty understanding how to deal 
with any of these memory representations. 


We close this section with an example of modified doubly linked ring struc- 
tures applied to a problem we have considered before: arithmetic on polynomials. 
Algorithm 2.2.4A performs the addition of one polynomial to another, given that 
the two polynomials are expressed as circular lists; various other algorithms in 
that section give other operations on polynomials. However, the polynomials 
of Section 2.2.4 are restricted to at most three variables. When multi-variable 
polynomials are involved, it is usually more appropriate to use a tree structure 
instead of a linear list. 

A polynomial either is a constant or has the form 


So ga, 


0<j<n 
where x is a variable, n > 0, 0 = eọ < ey < +--+: < en, and go,..-,9n are 
polynomials involving only variables alphabetically less than x; g1,..., gn are not 


zero. This recursive definition of polynomials lends itself to tree representation 
as indicated in Fig. 28. Nodes have six fields, which in the case of MIX might fit 
in three words: 


+ | O | LEFT | RIGHT 


+ |EXP} UP DOWN (17) 


Here LEFT, RIGHT, UP, and DOWN are links; EXP is an integer representing an 
exponent; and CV is either a constant (coefficient) or the alphabetic name of a 
variable. The root node has UP = A, EXP = 0, LEFT = RIGHT = + (self). 

The following algorithm illustrates traversal, insertion, and deletion in such 
a four-way-linked tree, so it bears careful study. 


Algorithm A (Addition of polynomials). This algorithm adds polynomial (P) to 
polynomial (Q), assuming that P and Q are pointer variables that link to the roots 
of distinct polynomial trees having the form shown in Fig. 28. At the conclusion 
of the algorithm, polynomial(P) will be unchanged, and polynomial(Q) will 
contain the sum. 


Al. [Test type of polynomial.| If DOWN(P) = A (that is, if P points to a 
constant), then set Q + DOWN(Q) zero or more times until DOWN(Q) = A 
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and go to A3. If DOWN (P) Æ A, then if DOWN(Q) = A or if CV (Q) < CV (P), 
go to A2. Otherwise if CV(Q) = CV (P), set P + DOWN (P), Q «+ DOWN (Q) 
and repeat this step; if CV (Q) > CV (P), set Q «+ DOWN (Q) and repeat this 
step. (Step A1 either finds two matching terms of the polynomials or 
else determines that an insertion of a new variable must be made into the 
current part of polynomial (Q).) 


[Downward insertion.] Set R <= AVAIL, S «+ DOWN(Q). If S # A, set 
UP(S) «+ R, S + RIGHT(S) and if EXP(S) Æ 0, repeat this operation 
until ultimately EXP(S) = 0. Set UP(R) + Q, DOWN(R) + DOWN(Q), 
LEFT(R) < R, RIGHT(R) + R, CV(R) + CV(Q) and EXP(R) & 0. Finally, 
set CV(Q) + CV(P) and DOWN(Q) «+ R, and return to Al. (We have inserted 
a “dummy” zero polynomial just below NODE(Q), to obtain a match with 
a corresponding polynomial found within P’s tree. The link manipulations 
done in this step are straightforward and may be derived easily using 
“before-and-after” diagrams, as explained in Section 2.2.3.) 

[Match found.] (At this point, P and Q point to corresponding terms of 
the given polynomials, so addition is ready to proceed.) Set CV(Q) + 
cV(Q) + CV(P). If this sum is zero and if EXP(Q) 4 0, go to step A8. If 
EXP (Q) = 0, go to A7. 


[Advance to left.] (After successfully adding a term, we look for the next 
term to add.) Set P «+ LEFT(P). If EXP(P) = 0, go to A6. Otherwise 
set Q + LEFT(Q) one or more times until EXP(Q) < EXP(P). If then 
EXP(Q) = EXP(P), return to step Al. 


[Insert to right.] Set R <= AVAIL. Set UP(R) < UP(Q), DOWN(R) <+ A, 
CV(R) «+ 0, LEFT(R) + Q, RIGHT(R) «+ RIGHT(Q), LEFT(RIGHT(R)) ¢ R, 
RIGHT(Q) + R, EXP(R) + EXP (P), and Q & R. Return to step Al. (We 
needed to insert a new term in the current row, just to the right of NODE (Q), 
in order to match a corresponding exponent in polynomial(P). As in 
step A2, a “before-and-after” diagram makes the operations clear.) 


Return upward.] (A row of polynomial(P) has now been completely tra- 
versed.) Set P < UP (P). 


Move Q up to right level.| If UP(P) = A, go to All; otherwise set 
Q «+ UP(Q) zero or more times until CVCUP(Q)) = CV(UP(P)). Return 
to step A4. 


Delete zero term.] Set R + Q, Q + RIGHT (R), S + LEFT(R), LEFT(Q) < S, 
RIGHT(S) + Q, and AVAIL +R. (Cancellation occurred, so a row element 
of polynomial(Q) is deleted.) If now EXP(LEFT(P)) = 0 and Q = S, go 
to A9; otherwise return to A4. 


[Delete constant polynomial.] (Cancellation has caused a polynomial to 
reduce to a constant, so a row of polynomial(Q) is deleted.) Set R < Q, 
Q + UP(Q), DOWN(Q) «+ DOWN(R), CV(Q) + CV(R), and AVAIL = R. Set 
S + DOWN(Q); if S Æ A, set UP(S) + Q, S + RIGHT(S), and if EXP (S) Æ 0, 
repeat this operation until ultimately EXP(S) = 0. 
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A10. [Zero detected?] If DOWN(Q) = A, CV(Q) = 0, and EXP(Q) ¥ 0, set P + 
UP (P) and go to A8; otherwise go to A6. 


A11. [Terminate.] Set Q < UP(Q) zero or more times until UP(Q) = A (thus 
bringing Q to the root of the tree). J 


This algorithm will actually run much faster than Algorithm 2.2.4A if poly- 
nomial(P) has few terms and polynomial(Q) has many, since it is not necessary 
to pass over all of polynomial(Q) during the addition process. The reader will 
find it instructive to simulate Algorithm A by hand, adding the polynomial 
xy — x? — xyz — z? + 3xz3 to the polynomial shown in Fig. 28. (This case does 
not demonstrate the efficiency of the algorithm, but it makes the algorithm go 
through all of its paces by showing the difficult situations that must be handled.) 
For further commentary on Algorithm A, see exercises 12 and 13. 

No claim is being made here that the representation shown in Fig. 28 is the 
“best” for polynomials in several variables; in Chapter 8 we will consider another 
format for polynomial representation, together with arithmetic algorithms using 
an auxiliary stack, with significant advantages of conceptual simplicity when 
compared to Algorithm A. Our main interest in Algorithm A is the way it 
typifies manipulations on trees with many links. 


EXERCISES 


1. [20] If we had only LTAG, INFO, and RTAG fields (not LLINK) in a level order 
sequential representation like (8), would it be possible to reconstruct the LLINKs? (In 
other words, are the LLINKs redundant in (8), as the RLINKs are in (3)?) 


2. [22] (Burks, Warren, and Wright, Math. Comp. 8 (1954), 53-57.) The trees (2) 
stored in preorder with degrees would be 


DEGREE 2010 3 10 100 
INFO ABCKDEHFIJIG 


[compare with (9), where postorder was used]. Design an algorithm analogous to 
Algorithm F to evaluate a locally defined function of the nodes by going from right to 
left in this representation. 


3. [24] Modify Algorithm 2.3.2D so that it follows the ideas of Algorithm F, placing 
the derivatives it computes as intermediate results on a stack, instead of recording their 
locations in an anomalous fashion as is done in step D3. (See exercise 2.3.2-21.) The 
stack may be maintained by using the RLINK field in the root of each derivative. 


4. [18] The trees (2) contain 10 nodes, five of which are terminal. Representation 
of these trees in the normal binary-tree fashion involves 10 LLINK fields and 10 RLINK 
fields (one for each node). Representation of these trees in the form (10), where LLINK 
and INFO share the same space in a node, requires 5 LLINKs and 15 RLINKs. There are 
10 INFO fields in each case. 

Given a forest with n nodes, m of which are terminal, compare the total number of 
LLINKs and RLINKs that must be stored using these two methods of tree representation. 


5. [16] A triply linked tree, as shown in Fig. 26, contains PARENT, LCHILD, and RLINK 
fields in each node, with liberal use of A-links when there is no appropriate node to 
mention in the PARENT, LCHILD, or RLINK field. Would it be a good idea to extend this 
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representation to a threaded tree, by putting “thread” links in place of the null LCHILD 
and RLINK entries, as we did in Section 2.3.1? 


6. [24] Suppose that the nodes of an oriented forest have three link fields, PARENT, 
LCHILD, and RLINK, but only the PARENT link has been set up to indicate the tree 
structure. The LCHILD field of each node is A and the RLINK fields are set as a linear 
list that simply links the nodes together in some order. The link variable FIRST points 
to the first node, and the last node has RLINK = A. 

Design an algorithm that goes through these nodes and fills in the LCHILD and 
RLINK fields compatible with the PARENT links, so that a triply linked tree representation 
like that in Fig. 26 is obtained. Also, reset FIRST so that it now points to the root of 
the first tree in this representation. 


7. [15] What classes would appear in (12) if the relation 9 = 3 had not been given 
in (11)? 


8. [15] Algorithm E sets up a tree structure that represents the given pairs of equiv- 
alent elements, but the text does not mention explicitly how the result of Algorithm E 
can be used. Design an algorithm that answers the question, “Is 7 = k?”, assuming 
that 1 <j <n,1<k<n, and that Algorithm E has set up the PARENT table for some 
set of equivalences. 


9. [20] Give a table analogous to (15) and a diagram analogous to (16) that shows 
the trees present after Algorithm E has processed all of the equivalences in (11) from 
left to right. 


10. [28] In the worst case, Algorithm E may take order n? steps to process n equiv- 
alences. Show how to modify the algorithm so that the worst case is not this bad. 


11. [24] (Equivalence declarations.) Several compiler languages, notably FORTRAN, 
provide a facility for overlapping the memory locations assigned to sequentially stored 
tables. The programmer gives the compiler a set of relations of the form X[j] = 
Y[k], which means that variable X[j + s] is to be assigned to the same location as 
variable Y[k + s] for all s. Each variable is also given a range of allowable subscripts: 
“ARRAY X[l:u]” means that space is to be set aside in memory for the table entries 
X(l], XE + 1], ..., X[u]. For each equivalence class of variables, the compiler reserves 
as small a block of consecutive memory locations as possible, to contain all the table 
entries for the allowable subscript values of these variables. 

For example, suppose we have ARRAY X[0:10], ARRAY Y[3:10], ARRAY A[1:1], and 
ARRAY Z[—2:0], plus the equivalences X[7] = Y[3], Z[0] = A[0], and Y[1] = A[8]. 
We must set aside 20 consecutive locations 


Xo X1 X2 X3 Xa X5 X6 X7 Xs XoXo 
e e e e e e e e e e e e e e e e e e e e 
Z-2Z—1 Zo Ai Y3 Ya Ys Ye Yz Ys Yo Yio 


for these variables. (The location following A[1] is not an allowable subscript value for 
any of the arrays, but it must be reserved anyway.) 

The object of this exercise is to modify Algorithm E so that it applies to the 
more general situation just described. Assume that we are writing a compiler for 
such a language, and the tables inside our compiler program itself have one node for 
each array, containing the fields NAME, PARENT, DELTA, LBD, and UBD. Assume that 
the compiler program has previously processed all the ARRAY declarations, so that if 
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ARRAY X[l:u] has appeared and if P points to the node for X, then 


NAME(P) = “X”, PARENT(P) =A, DELTA(P) =0, 
LBD(P) =l, UBD(P) =u. 
The problem is to design an algorithm that processes the equivalence declarations, so 
that, after this algorithm has been performed, 


PARENT(P) = A means that locations X[LBD(P)],...,X[UBD(P)] are to be reserved 
in memory for this equivalence class; 


PARENT(P) = Q Æ A means that location X[k] equals location Y[k + DELTA(P)], 
where NAME(Q) = “Y”. 


For example, before the equivalences listed above we might have the nodes 


P NAME(P) PARENT (P) DELTA(P) LBD (P) UBD(P) 


a x A 0 0 10 
B Y A 0 3 10 
y A A 0 1 1 
ô Z A 0 -2 


After the equivalences are processed, the nodes might appear thus: 


Q X A * —5 14 
B Y a 4 * * 
y A ô 0 * * 
ô Z a —3 * * 


(“*” denotes irrelevant information.) 

Design an algorithm that makes this transformation. Assume that inputs to your 

algorithm have the form (P, j, Q, k), denoting X[j] = Y[k], where NAME(P) = “X” and 
NAME(Q) = “Y”. Be sure to check whether the equivalences are contradictory; for 
example, X[1] = Y[2] contradicts X[2] = Y[1]. 
12. [21] At the beginning of Algorithm A, the variables P and Q point to the roots of 
two trees. Let Po and Qo denote the values of P and Q before execution of Algorithm A. 
(a) After the algorithm terminates, is Qo always the address of the root of the sum of 
the two given polynomials? (b) After the algorithm terminates, have P and Q returned 
to their original values Po and Qo? 


13. [M29] Give an informal proof that at the beginning of step A8 of Algorithm A we 
always have EXP(P) = EXP(Q) and CV(UP(P)) = CV(UP(Q)). (This fact is important to 
the proper understanding of that algorithm.) 


14. [40] Give a formal proof (or disproof) of the validity of Algorithm A. 


15. [40] Design an algorithm to compute the product of two polynomials represented 
as in Fig. 28. 


16. [M24] Prove the validity of Algorithm F. 


17. [25] Algorithm F evaluates a “bottom-up” locally defined function, namely, one 
that should be evaluated at the children of a node before it is evaluated at the node. 
A “top-down” locally defined function f is one in which the value of f at a node x 
depends only on x and the value of f at the parent of x. Using an auxiliary stack, 
design an algorithm analogous to Algorithm F that evaluates a “top-down” function f 
at each node of a tree. (Like Algorithm F, your algorithm should work efficiently on 
trees that have been stored in postorder with degrees, as in (9).) 
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> 18. [28] Design an algorithm that, given the two tables INFO1[j] and RLINK[j] for 
1 < j < n corresponding to preorder sequential representation, forms tables INF02 [j] 
and DEGREE[j] for 1 < j < n, corresponding to postorder with degrees. For example, 
according to (3) and (9), your algorithm should transform 
j 1 2 3 4 5 6 7 8 9 10 
INFO1[7] A B C K D E H F G 
RLINK[J] 5 3 0 0 0 8 0 10 0 0 


S 


into 
INF02[j] B K C A H E J F G OD 
DEGREE[ j] 0 0 1 2 0 1 0 1 0 3 


19. [M27] Instead of using SCOPE links in (5), we could simply list the number of 
descendants of each node, in preorder: 
DESC 3 0 10510100 
INFO ABCKDEHFJG 
Let dıd2...dn be the sequence of descendant numbers of a forest, obtained in this way. 
a) Show that k+d, < n for 1 < k < n, and that k < j < k+dp implies j+d; < k+dp. 
b) Conversely, prove that if dıd2 .. . dn is a sequence of nonnegative integers satisfying 
the conditions of (a), it is the sequence of descendant numbers of a forest. 
c) Suppose dıdə...dn and did} ...dh are the descendant number sequences for two 
forests. Prove that there is a third forest whose descendant numbers are 


min(d;, d1) min(dz, d2) . . . min (dn, dm). 


2.3.4. Basic Mathematical Properties of Trees 


Tree structures have been the object of extensive mathematical investigations for 
many years, long before the advent of computers, and many interesting facts have 
been discovered about them. In this section we will survey the mathematical 
theory of trees, which not only gives us more insight into the nature of tree 
structures but also has important applications to computer algorithms. 

Nonmathematical readers are advised to skip to subsection 2.3.4.5, which 
discusses several topics that arise frequently in the applications we shall study 
later. 

The material that follows comes mostly from a larger area of mathematics 
known as the theory of graphs. Unfortunately, there will probably never be 
a standard terminology in this field, and so the author has followed the usual 
practice of contemporary books on graph theory, namely to use words that are 
similar but not identical to the terms used in any other books on graph theory. 
An attempt has been made in the following subsections (and, indeed, throughout 
this book) to choose short, descriptive words for the important concepts, selected 
from those that are in reasonably common use and that do not sharply conflict 
with other common terminology. The nomenclature used here is also biased 
towards computer applications. Thus, an electrical engineer may prefer to call a 
“tree” what we call a “free tree”; but we want the shorter term “tree” to stand for 
the concept that is generally used in the computer literature and that is so much 
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E 
C D 
Fig. 29. A graph. Fig. 30. A free tree. 


more important in computer applications. If we were to follow the terminology 
of some authors on graph theory, we would have to say “finite labeled rooted 
ordered tree” instead of just “tree,” and “topological bifurcating arborescence” 
instead of “binary tree”! 


2.3.4.1. Free trees. A graph is generally defined to be a set of points (called 
vertices) together with a set of lines (called edges) joining certain pairs of distinct 
vertices. There is at most one edge joining any pair of vertices. Two vertices are 
called adjacent if there is an edge joining them. If V and V’ are vertices and if 
n > 0, we say that (Vo,Vi,...,V,) is a walk of length n from V to V’ if V = VW, 
Vp is adjacent to Vk+1 for 0 < k < n, and Vp = V’. The walk is a path if vertices 
Vo, Vi, ---,; Vn are distinct; it is a cycle if Vo through V,,_; are distinct, Vn = Vo, 
and n > 3. Sometimes we are less precise, and refer to a cycle as “a path from a 
vertex to itself.” We often speak of a “simple path” to emphasize the fact that 
we’re talking about a path instead of an arbitrary walk. A graph is connected if 
there is a path between any two vertices of the graph. 

These definitions are illustrated in Fig. 29, which shows a connected graph 
with five vertices and six edges. Vertex C is adjacent to A but not to B; there 
are two paths of length two from B to C, namely (B, A, C) and (B, D, C). There 
are several cycles, including (B, D, FE, B). 

A free tree or “unrooted tree” (Fig. 30) is defined to be a connected graph 
with no cycles. This definition applies to infinite graphs as well as to finite ones, 
although for computer applications we naturally are most concerned with finite 
trees. There are many equivalent ways to define a free tree; some of them appear 
in the following well-known theorem: 


Theorem A. If G is a graph, the following statements are equivalent: 
a) G is a free tree. 


b) G is connected, but if any edge is deleted, the resulting graph is no longer 
connected. 


c) If V and V’ are distinct vertices of G, there is exactly one simple path from 
V to V’. 


Furthermore, if G is finite, containing exactly n > 0 vertices, the following 
statements are also equivalent to (a), (b), and (c): 


d) G contains no cycles and has n — 1 edges. 
e) G is connected and has n — 1 edges. 
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Proof. (a) implies (b), for if the edge V — V” is deleted but G is still connected, 
there must be a simple path (V, V1,..., V’) of length two or more—see exer- 
cise 2—and then (V, Vi, ..., V’, V) would be a cycle in G. 

(b) implies (c), for there is at least one simple path from V to V’. And if 
there were two such paths (V,Vi,...,V’) and (V,V/,...,V’), we could find the 
smallest k for which V, 4 vi ; deleting the edge Vk—1 V; would not disconnect 
the graph, since there would still be a path (V,_,,Vj,...,V’,...,V,) from Vk—1 
to V, that does not use the deleted edge. 

(c) implies (a), for if G contains a cycle (V,Vi,...,V), there are two simple 
paths from V to Vj. 

To show that (d) and (e) are also equivalent to (a), (b), and (c), let us first 
prove an auxiliary result: If G is any finite graph that has no cycles and at least 
one edge, then there is at least one vertex that is adjacent to exactly one other 
vertex. This follows because we can find some vertex V; and an adjacent vertex 
V2; for k > 2 either Vp is adjacent to Vk—ı and no other, or it is adjacent to a 
vertex that we may call Vk+1 Æ Vk—1. Since there are no cycles, Vi, V2,..., Vk+1 
must be distinct vertices, so this process must ultimately terminate. 

Now assume that G is a free tree with n > 1 vertices, and let Vp be a vertex 
that is adjacent to only one other vertex, namely V,,_;. If we delete V,, and the 
edge Vn—1— Vn, the remaining graph G” is a free tree, since Vp, appears in no 
simple path of G except as the first or the last element. This argument proves 
(by induction on n) that G has n — 1 edges; hence (a) implies (d). 

Assume that G satisfies (d) and let Vn, Vn—-1, G’ be as in the preceding 
paragraph. Then the graph G is connected, since Vp is connected to V,—1, 
which (by induction on n) is connected to all other vertices of G”. Thus (d) 
implies (e). 

Finally assume that G satisfies (e). If G contains a cycle, we can delete any 
edge appearing in that cycle and G would still be connected. We can therefore 
continue deleting edges in this way until we obtain a connected graph G” with 
n — 1 — k edges and no cycles. But since (a) implies (d), we must have k = 0, 
that is, G=G" J 


The idea of a free tree can be applied directly to the analysis of computer 
algorithms. In Section 1.3.3, we discussed the application of Kirchhoft’s first 
law to the problem of counting the number of times each step of an algorithm 
is performed; we found that Kirchhoff’s law does not completely determine the 
number of times each step is executed, but it reduces the number of unknowns 
that must be specially interpreted. The theory of trees tells us how many 
independent unknowns will remain, and it gives us a systematic way to find them. 

It is easier to understand the method that follows if an example is studied, 
so we will work an example as the theory is being developed. Figure 31 shows an 
abstracted flow chart for Program 1.3.3A, which was subjected to a “Kirchhoff’s 
law” analysis in Section 1.3.3. Each box in Fig. 31 represents part of the 
computation, and the letter or number inside the box denotes the number of 
times that computation will be performed during one run of the program, using 
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Fig. 31. Abstracted flow chart of Program 1.3.3A. 


the notation of Section 1.3.3. An arrow between boxes represents a possible 
jump in the program. The arrows have been labeled e1, €2,...,¢€27. Our goal is 
to find all relations between the quantities A, B, C, D, E, F, G, H, J, K, L, P, 
Q, R, and S that are implied by Kirchhoff’s law, and at the same time we hope 
to gain some insight into the general problem. (Note: Some simplifications have 
already been made in Fig. 31; for example, the box between C and E has been 
labeled “1”, and this in fact is a consequence of Kirchhoff’s law.) 

Let Ej denote the number of times branch e; is taken during the execution 
of the program being studied; Kirchhoff’s law is 


sum of E’s into box = value in box = sum of F’s leaving box; (1) 
for example, in the case of the box marked K we have 
Eig + E2 = K = Eis + Fa). (2) 


In the discussion that follows, we will regard E1, E2,..., 27 as the unknowns, 
instead of A, B,...,S. 

The flow chart in Fig. 31 may be abstracted further so that it becomes 
a graph G as in Fig. 32. The boxes have shrunk to vertices, and the arrows 
€1,€2,... now represent edges of the graph. (A graph, strictly speaking, has 
no implied direction in its edges, and the direction of the arrows should be 
ignored when we refer to graph-theoretical properties of G. Our application 
to Kirchhoff’s law, however, makes use of the arrows, as we will see shortly.) 
For convenience an extra edge eg has been drawn from the Stop vertex to the 
Start vertex, so that Kirchhoff’s law applies uniformly to all parts of the graph. 
Figure 32 also includes some other minor changes from Fig. 31: An extra vertex 
and edge have been added to divide e13 into two parts e13 and ef}, so that the 
basic definition of a graph (no two edges join the same two vertices) is valid; e19 
has also been split up in this way. A similar modification would have been made 
if we had any vertex with an arrow leading back to itself. 

Some of the edges in Fig. 32 have been drawn much heavier than the others. 
These edges form a free subtree of the graph, connecting all the vertices. It 
is always possible to find a free subtree of the graphs arising from flow charts, 
because the graphs must be connected and, by part (b) of Theorem A, if G 
is connected and not a free tree, we can delete some edge and still have the 
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Fig. 32. Graph corresponding to Fig. 31, including a free subtree. 


resulting graph connected; this process can be iterated until we reach a free 
subtree. Another algorithm for finding a free subtree appears in exercise 6. We 
can in fact always discard the edge eg (which went from the Stop to the Start 
vertex) first; thus we may assume that ep does not appear in the subtree chosen. 

Let G’ be a free subtree of the graph G found in this way, and consider 
any edge V — V’ of G that is not in G. We may now note an important 
consequence of Theorem A: G” plus this new edge V — V” contains a cycle; and 
in fact there is exactly one cycle, having the form (V,V’,...,V), since there is 
a unique simple path from V’ to V in G’. For example, if G” is the free subtree 
shown in Fig. 32, and if we add the edge e2, we obtain a cycle that goes along 
e2 and then (in the direction opposite to the arrows) along e4 and e3. This cycle 
may be written algebraically as “e2 — e4 — e3”, using plus signs and minus signs 
to indicate whether the cycle goes in the direction of the arrows or not. 

If we carry out this process for each edge not in the free subtree, we obtain 
the so-called fundamental cycles, which in the case of Fig. 32 are 


Co: eo + e1 + e3 + e4 + eg +e7 + €9 + €10 + C11 + C12 + €14, 
Ca: €2 — €4— es, 

C5: es — ey — €e, 

Cg: eg +e3 +e4+ e6 + €7, 

Cis: e13 + e12 + €13, 


C17: €17 + €22 + €24 + €27 + €11 + €15 + E16, 


ve, I 1 
Cig: Eig + €1g + €g, 


C20: e20 + e18 + €22 + €23, 


C21: e21 — €16 — €15 — €11 — €27 — €24 — €22 €18, 


C25: €25 + e26 — €27- 
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Obviously an edge e; that is not in the free subtree will appear in only one of 
the fundamental cycles, namely C}. 

We are now approaching the climax of this construction. Each fundamental 
cycle represents a solution to Kirchhoff’s equations; for example, the solution 
corresponding to C% is to let Ey = +1, E4 = —1, E3 = —1, and all other E’s = 0. 
It is clear that flow around a cycle in a graph always satisfies the condition 
(1) of Kirchhoff’s law. Moreover, Kirchhoff’s equations are “homogeneous,” 
so the sum or difference of solutions to (1) yields another solution. Therefore 
we may conclude that the values of Eo, E2, Es,..., E25 are independent in the 
following sense: 


If £0, £2, . . . , £25 are any real numbers (one x; for each e; not in the free 
subtree G’), there is a solution to Kirchhoff’s equations (1) such that (4) 
Eo = 20, E2 = £2, ..., E25 = T25. 


Such a solution is found by going zo times around the cycle Co, x2 times around 
cycle Cy, etc. Furthermore, we find that the values of the remaining variables 
Eı, E3, E4,... are completely dependent on the values Eo, E2,..., E25: 


The solution mentioned in statement (4) is unique. (5) 


For if there are two solutions to Kirchhoff’s equations such that Ey = 2o,..., 
E25 = £25, we can subtract one from the other and we thereby obtain a solution 
in which Kp = Eo = Es =--- = E25 = 0. But now all Ej must be zero, for it is 
easy to see that a nonzero solution to Kirchhoff’s equations is impossible when 
the graph is a free tree (see exercise 4). Therefore the two assumed solutions 
must be identical. We have now proved that all solutions of Kirchhoff’s equations 
may be obtained as sums of multiples of the fundamental cycles. 

When these remarks are applied to the graph in Fig. 32, we obtain the 
following general solution of Kirchhoff’s equations in terms of the independent 
variables Eo, Ez, ssy Eos: 


Fy Eo, Ei, = Eo, 

E3 = Eo — E2 + Es, Eis = Eir — Fai, 

E, = Eo — Ez + Es, Eis = Fiz — Ea, 

Es = Eo — Es + Es, Fig = Eg + E2 — Ear, 

By = Eo- E; + Es, Eig = Eib, (6) 
Eg = Eo, E2 = Fiz + Ez — Eon, 

Eio = Eo, E23 = E20, 

Ei = Eo + Fiz — Ea, Ez, = Eir — Ez, 

Ei = Eo + Ej, Figg = E, 

Eig = Ejs, E7 = Ey7 — Eo — E25. 


To obtain these equations, we merely list, for each edge ej in the subtree, all Ek 
for which ej appears in cycle Cy, with the appropriate sign. [Thus, the matrix 
of coefficients in (6) is just the transpose of the matrix of coefficients in (3).] 
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Strictly speaking, Co should not be called a fundamental cycle, since it 
involves the special edge eg. We may call Co minus the edge eo a fundamental 
path from Start to Stop. Our boundary condition, that the Start and Stop boxes 
in the flow chart are performed exactly once, is equivalent to the relation 


Fo =1. (7) 


The preceding discussion shows how to obtain all solutions to Kirchhoff’s 
law; the same method may be applied (as Kirchhoff himself applied it) to 
electrical circuits instead of program flow charts. It is natural to ask at this 
point whether Kirchhoff’s law is the strongest possible set of equations that can 
be given for the case of program flow charts, or whether more can be said: Any 
execution of a computer program that goes from Start to Stop gives us a set of 
values E1, F2,..., E27 for the number of times each edge is traversed, and these 
values obey Kirchhoft’s law; but are there solutions to Kirchhoff’s equations that 
do not correspond to any computer program execution? (In this question, we do 
not assume that we know anything about the given computer program, except 
its flow chart.) If there are solutions that meet Kirchhoff’s conditions but do not 
correspond to actual program execution, we can give stronger conditions than 
Kirchhoff’s law. For the case of electrical circuits Kirchhoff himself gave a second 
law [Ann. Physik und Chemie 64 (1845), 497-514]: The sum of the voltage drops 
around a fundamental cycle must be zero. This second law does not apply to 
our problem. 

There is indeed an obvious further condition that the E’s must satisfy, if 
they are to correspond to some actual walk in the flow chart from Start to Stop; 
they must be integers, and in fact they must be nonnegative integers. This is 
not a trivial condition, since we cannot simply assign any arbitrary nonnegative 
integer values to the independent variables £2, Es,..., E25; for example, if we 
take E2 = 2 and Eg = 0, we find from (6) and (7) that E3 = —1. (Thus, 
no execution of the flow chart in Fig. 31 will take branch e> twice without 
taking branch eg at least once.) The condition that all the E’s be nonnegative 
integers is not enough either; for example, consider the solution in which Ey = 1, 
Ez Es PES Eiz E0 E1 Eos 0; there is no way to get to €18 
except via e15. The following condition is a necessary and sufficient condition 
that answers the problem raised in the previous paragraph: Let E2, Es,..., E25 
be any given values, and determine E4, F3,..., £27 according to (6), (7). Assume 
that all the E’s are nonnegative integers, and assume that the graph whose edges 
are those e; for which Æ; > 0, and whose vertices are those that touch such ej, is 
connected. Then there is a walk from Start to Stop in which edge e; is traversed 
exactly Ej times. This fact is proved in the next section (see exercise 2.3.4.2-24). 

Let us now summarize the preceding discussion: 


Theorem K. Ifa flow chart (such as Fig. 31) contains n boxes (including Start 
and Stop) and m arrows, it is possible to find m — n+ 1 fundamental cycles 
and a fundamental path from Start to Stop, such that any walk from Start 
to Stop is equivalent (in terms of the number of times each edge is traversed) 
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to one traversal of the fundamental path plus a uniquely determined number 
of traversals of each of the fundamental cycles. (The fundamental path and 
fundamental cycles may include some edges that are to be traversed in a direction 
opposite that shown by the arrow on the edge; we conventionally say that such 
edges are being traversed —1 times.) 

Conversely, for any traversal of the fundamental path and the fundamental 
cycles in which the total number of times each edge is traversed is nonnegative, 
and in which the vertices and edges corresponding to a positive number of 
traversals form a connected graph, there is at least one equivalent walk from 
Start to Stop. I 

The fundamental cycles are found by picking a free subtree as in Fig. 32; if 
we choose a different subtree we get, in general, a different set of fundamental 
cycles. The fact that there are m — n + 1 fundamental cycles follows from 
Theorem A. The modifications we made to get from Fig. 31 to Fig. 32, after 
adding eg, do not change the value of m — n + 1, although they may increase 
both m and n; the construction could have been generalized so as to avoid these 
trivial modifications entirely (see exercise 9). 

Theorem K is encouraging because it says that Kirchhoff’s law (which con- 
sists of n equations in the m unknowns F}), E2,..., Em) has just one “redun- 
dancy”: These n equations allow us to eliminate n — 1 unknowns. However, the 
unknown variables throughout this discussion have been the number of times the 
edges have been traversed, not the number of times each box of the flow chart 
has been entered. Exercise 8 shows how to construct another graph whose edges 
correspond to the boxes of the flow chart, so that the theory above can be used 
to deduce the true number of redundancies between the variables of interest. 

Applications of Theorem K to software for measuring the performance of 
programs in high-level languages are discussed by Thomas Ball and James R. 
Larus in ACM Trans. Prog. Languages and Systems 16 (1994), 1319-1360. 


EXERCISES 

1. [14] List all cycles from B to B that are present in the graph of Fig. 29. 

2. [M20] Prove that if V and V’ are vertices of a graph and if there is a walk from 
V to V’, then there is a (simple) path from V to V”. 

3. [15] What walk from Start to Stop is equivalent (in the sense of Theorem K) to 
one traversal of the fundamental path plus one traversal of cycle C2 in Fig. 32? 

4. [M20] Let G’ be a finite free tree in which arrows have been drawn on its edges 
€1,---,;€n—1; let E1,...,£,—-1 be numbers satisfying Kirchhoff’s law (1) in G”. Show 
that Ey =---= En-1 =0. 

5. [20] Using Eqs. (6), express the quantities A, B,...,S that appear inside the 
boxes of Fig. 31 in terms of the independent variables Eo, Es,..., E25. 


6. [M27] Suppose a graph has n vertices Vi,...,Vn and m edges e€1,...,@m. Each 
edge e is represented by a pair of integers (a,b) if it joins Va to Vs. Design an algorithm 
that takes the input pairs (a1,b1), ..., (am, bm) and prints out a subset of edges that 
forms a free tree; the algorithm reports failure if this is impossible. Strive for an efficient 
algorithm. 
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7. [22] Carry out the construction in the text for the flow chart 


Start 


e9 


using the free subtree consisting of edges e1, e2, €3, €4, eg. What are the fundamental 
cycles? Express E1, Eo, E3, E4, Eg in terms of Es, Eg, E7, and Es. 


> 8. [M25] When applying Kirchhoff’s first law to program flow charts, we usually are 
interested only in the vertex flows (the number of times each box of the flow chart 
is performed), not the edge flows analyzed in the text. For example, in the graph of 
exercise 7, the vertex flows are A = Ez + E4, B= Es, C= E3 + E7+ Es, D = Es + Eo. 
If we group some vertices together, treating them as one “supervertex,” we can 
combine edge flows that correspond to the same vertex flow. For example, edges e2 
and e4 can be combined in the flow chart above if we also put B with D: 


(Here eo has also been added from Stop to Start, as in the text.) Continuing this 
procedure, we can combine e3 + e7, then (e3 + e7) + eg, then eg + e9, until we obtain 
the reduced flow chart having edges s = e1, a = e2 + €4, b = €5, C = €3 + e7 + 6s, 
d = es + e9, t = eo, precisely one edge for each vertex in the original flow chart: 


By construction, Kirchhoff’s law holds in this reduced flow chart. The new edge 
flows are the vertex flows of the original; hence the analysis in the text, applied to the 
reduced flow chart, shows how the original vertex flows depend on each other. 

Prove that this reduction process can be reversed, in the sense that any set of 
flows {a,b,...} satisfying Kirchhoff’s law in the reduced flow chart can be “split up” 
into a set of edge flows {eo,e1,...} in the original flow chart. These flows e; satisfy 
Kirchhoff’s law and combine to yield the given flows {a,b,...}; some of them might, 
however, be negative. (Although the reduction procedure has been illustrated here for 
only one particular flow chart, your proof should be valid in general.) 


9. [M22] Edges e13 and e19 were split into two parts in Fig. 32, since a graph is not 
supposed to have two edges joining the same two vertices. However, if we look at the 
final result of the construction, this splitting into two parts seems quite artificial since 
Eis = Et’; and Eig = Ejh are two of the relations found in (6), while E73 and Ejh are 
two of the independent variables. Explain how the construction could be generalized 
so that an artificial splitting of edges may be avoided. 
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10. [16] An electrical engineer, designing the circuitry for a computer, has n terminals 
Tı, T2,...,Tn that should be at essentially the same voltage at all times. To achieve 
this, the engineer can solder wires between any pairs of terminals; the idea is to make 
enough wire connections so that there is a path through the wires from any terminal to 
any other. Show that the minimum number of wires needed to connect all the terminals 
is n — 1, and n — 1 wires achieve the desired connection if and only if they form a free 
tree (with terminals and wires standing for vertices and edges). 


11. [M27] (R. C. Prim, Bell System Tech. J. 36 (1957), 1389-1401.) Consider the 
wire connection problem of exercise 10 with the additional proviso that a cost c(i, 7) is 
given for each i < j, denoting the expense of wiring terminal T; to terminal Tj. Show 
that the following algorithm gives a connection tree of minimum cost: “If n = 1, do 
nothing. Otherwise, renumber terminals {1,...,2—1} and the associated costs so that 
e(n — 1, n) = mini<ien c(i, n); connect terminal T,—1 to Tn; then change c(j, n — 1) 
to min(c(j, n — 1), e(j,n)) for 1 < j < n — 1, and repeat the algorithm for n — 1 
terminals T;,...,7n—1 using these new costs. (The algorithm is to be repeated with 
the understanding that whenever a connection is subsequently requested between the 
terminals now called Tj and T;,,_1, the connection is actually made between terminals 
now called T; and Tn if it is cheaper; thus T,-1 and Tn are being regarded as though 
they were one terminal in the remainder of the algorithm.)” This algorithm may also 
be stated as follows: “Choose a particular terminal to start with; then repeatedly make 
the cheapest possible connection from an unchosen terminal to a chosen one, until all 
have been chosen.” 


Fig. 33. Free tree 1 "Ts 
of minimum cost. n To 
(See exercise 11.) 0 1 2 3 4 5 


For example, consider Fig. 33(a), which shows nine terminals on a grid; let the 
cost of connecting two terminals be the wire length, namely the distance between them. 
(The reader may wish to try to find a minimal cost tree by hand, using intuition instead 
of the suggested algorithm.) The algorithm would first connect Tg to Ty, then Te to 
Ts, T5 to Te, To to Te, Tı to T2, Ts to Tı, T7 to T3, and finally T4 to either To or Te. 
A minimum cost tree (wire length 7 + 2 V2 + 2 v5) is shown in Fig. 33(b). 


12. [29] The algorithm of exercise 11 is not stated in a fashion suitable for direct 
computer implementation. Reformulate that algorithm, specifying in more detail the 
operations that are to be done, in such a way that a computer program can carry out 
the process with reasonable efficiency. 


13. [M24] Consider a graph with n vertices and m edges, in the notation of exercise 6. 
Show that it is possible to write any permutation of the integers {1,2,...,n} as 
a product of transpositions (ak;bk;)(akzbka).-- (Gn, be,) if and only if the graph is 
connected. (Hence there are sets of n — 1 transpositions that generate all permutations 
on n elements, but no set of n — 2 will do so.) 
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2.3.4.2. Oriented trees. In the previous section, we saw that an abstracted 
flow chart may be regarded as a graph, if we ignore the direction of the arrows 
on its edges; the graph-theoretic ideas of cycle, free subtree, etc., were shown to 
be relevant in the study of flow charts. There is a good deal more that can be 
said when the direction of each edge is given more significance, and in this case 
we have what is called a “directed graph” or “digraph.” 

Let us define a directed graph formally as a set of vertices and a set of arcs, 
each arc leading from a vertex V to a vertex V”. If e is an arc from V to V’ 
we say V is the initial vertex of e, and V’ is the final vertex, and we write 
V = init(e), V’ = fin(e). The case that init(e) = fin(e) is not excluded (although 
it was excluded from the definition of edge in an ordinary graph), and several 
different arcs may have the same initial and final vertices. The out-degree of a 
vertex V is the number of arcs leading out from it, namely the number of arcs e 
such that init(e) = V; similarly, the in-degree of V is defined to be the number 
of arcs with fin(e) = V. 

The concepts of paths and cycles are defined for directed graphs in a manner 
similar to the corresponding definitions for ordinary graphs, but some important 
new technicalities must be considered. If e1,e2,...,e€n are arcs (with n > 1), 
we say that (€1, €2,...,@n) is an oriented walk of length n from V to V” if V = 
init(e1), V’ = fin(e,), and fin(e,) = init(e,41) for 1 < k < n. An oriented walk 
(€1, €2,---,€n) is called simple if init(e1), ..., init(e,) are distinct and fin(e1), 
..., fin(e,) are distinct; such a walk is an oriented cycle if fin(e,) = init(e1), 
otherwise it’s an oriented path. (An oriented cycle can have length 1 or 2, but 
such short cycles were excluded from our definition of “cycle” in the previous 
section. Can the reader see why this makes sense?) 

As examples of these straightforward definitions, we may refer to Fig. 31 in 
the previous section. The box labeled “J” is a vertex with in-degree 2 (because 
of the arcs e16, €21) and out-degree 1. The sequence (e17, €19, €18, €22) is an 
oriented walk of length 4 from J to P; this walk is not simple since, for example, 
init(e19) = L = init(e22). The diagram contains no oriented cycles of length 1, 
but (e1g, €19) is an oriented cycle of length 2. 

A directed graph is said to be strongly connected if there is an oriented path 
from V to V’ for any two vertices V 4 V’. It is said to be rooted if there is at 
least one root, that is, at least one vertex R such that there is an oriented path 
from V to R for all V4 R. “Strongly connected” always implies “rooted,” but 
the converse does not hold. A flow chart such as Fig. 31 in the previous section 
is an example of a rooted digraph, with R the Stop vertex; with the additional 
arc from Stop to Start (Fig. 32) it becomes strongly connected. 

Every directed graph G corresponds in an obvious manner to an ordinary 
graph Go, if we ignore orientations and discard duplicate edges or loops. For- 
mally speaking, Go has an edge from V to V” if and only if V4 V’ and G has an 
arc from V to V’ or from V’ to V. We can speak of (unoriented) paths and cycles 
in G with the understanding that these are paths and cycles of Go; we can say 
that G is connected — this is a much weaker property than “strongly connected,” 
even weaker than “rooted” —if the corresponding graph Go is connected. 


2.3.4.2 ORIENTED TREES 373 


An oriented tree (see Fig. 34), sometimes called 
a “rooted tree” by other authors, is a directed graph R 
with a specified vertex R such that: 


a) Each vertex V Æ R is the initial vertex of exactly 
one arc, denoted by e[V]. 


b) R is the initial vertex of no arc; 


c) R is a root in the sense defined above (that is, 
for each vertex V Æ R there is an oriented path 
from V to R). 


It follows immediately that for each vertex VA R 
there is a unique oriented path from V to R; and Fig. 34. An oriented tree. 
hence there are no oriented cycles. 

Our previous definition of “oriented tree” (at the beginning of Section 2.3) 
is easily seen to be compatible with the new definition just given, when there 
are finitely many vertices. The vertices correspond to nodes, and the arc e[V] is 
the link from V to PARENT[V]. 

The (undirected) graph corresponding to an oriented tree is connected, 
because of property (c). Furthermore, it has no cycles. For if (Vo,Vi,-.--; Vn) 
is an undirected cycle with n > 3, and if the edge between Vo and V; is e[Vj], 
then the edge between V; and V2 must be e[Va], and similarly the edge between 
V,.—-1 and Vk must be e[V;] for 1 < k < n, contradicting the absence of oriented 
cycles. If the edge between Vo and V; is not e[V;], it must be e[Vo], and the same 
argument applies to the cycle 


(Vi, Vo, Vata ees Va), 


because V,, = Vo. Therefore an oriented tree is a free tree when the direction of 
the arcs is neglected. 

Conversely, it is important to note that we can reverse the process just 
described. If we start with any nonempty free tree, such as that in Fig. 30, we 
can choose any vertex as the root R, and assign directions to the edges. The 
intuitive idea is to “pick up” the graph at vertex R and shake it; then assign 
upward-pointing arrows. More formally, the rule is this: 


Change the edge V — V’ to an arc from V to V’ if and only if the simple 
path from V to R leads through V’, that is, if it has the form (Vo, Vi,..., Vn), 
where n > 0, Vo = V, Vy = V} Va =R. 


To verify that such a construction is valid, we need to prove that each edge 
V — V” is assigned the direction V <— V’ or the direction V — V’; and this 
is easy to prove, for if (V, V1, ..., R) and (V’,V/,...,R) are simple paths, there 
is a cycle unless V = V/ or V} = V’. This construction demonstrates that the 
directions of the arcs in an oriented tree are completely determined if we know 
which vertex is the root, so they need not be shown in diagrams when the root 
is explicitly indicated. 
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Fig. 35. Three tree structures. 


We now see the relation between three types of trees: the (ordered) tree, 
which is of principal importance in computer programs, as defined at the be- 
ginning of Section 2.3; the oriented tree (or unordered tree); and the free tree. 
Both of the latter two types arise in the study of computer algorithms, but not 
as often as the first type. The essential distinction between these types of tree 
structure is merely the amount of information that is taken to be relevant. For 
example, Fig. 35 shows three trees that are distinct if they are considered as 
ordered trees (with root at the top). As oriented trees, the first and second are 
identical, since the left-to-right order of subtrees is immaterial; as free trees, all 
three graphs in Fig. 35 are identical, since the root is immaterial. 


An Eulerian trail in a directed graph is an oriented walk (e1, €2,...,€m) such 
that every arc in the directed graph occurs exactly once, and fin(em) = init(e1). 
This is a “complete traversal” of the arcs of the digraph. (Eulerian trails get 
their name from Leonhard Euler’s famous discussion in 1736 of the impossibility 
of traversing each of the seven bridges in the city of Königsberg exactly once 
during a Sunday stroll. He treated the analogous problem for undirected graphs. 
Eulerian trails should be distinguished from “Hamiltonian cycles,” which are 
oriented cycles that encounter each verter exactly once; see Chapter 7.) 


A directed graph is said to be balanced (see Fig. 36) if every vertex V has 
the same in-degree as its out-degree, that is, if there are just as many edges 
with V as their initial vertex as there are with V as their final vertex. This 
condition is closely related to Kirchhoff’s law (see exercise 24). If a directed 
graph has an Eulerian trail, it must obviously be connected and balanced — 
unless it has isolated vertices, which are vertices with in-degree and out-degree 
both equal to zero. 

So far in this section we’ve looked at quite a few definitions (directed graph, 
arc, initial vertex, final vertex, out-degree, in-degree, oriented walk, oriented 
path, oriented cycle, oriented tree, Eulerian trail, isolated vertex, and the prop- 
erties of being strongly connected, rooted, and balanced), but there has been 
a scarcity of important results connecting these concepts. Now we are ready 
for meatier material. The first basic result is a theorem due to I. J. Good 
[J. London Math. Soc. 21 (1947), 167-169], who showed that Eulerian trails are 
always possible unless they are obviously impossible: 
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Fig. 36. A balanced directed graph. 


Theorem G. A finite, directed graph with no isolated vertices possesses an 
Eulerian trail if and only if it is connected and balanced. 


Proof. Assume that G is balanced, and let 
P = (e,.--,€m) 


be an oriented walk of longest possible length that uses no arc twice. Then if 
V = fin(em), and if k is the out-degree of V, all k arcs e with init(e) = V must 
already appear in P; otherwise we could add e and get a longer walk. But if 
init(e;) = V and j > 1, then fin(e;_1) = V. Hence, since G is balanced, we must 
have 


init(e1) = V = fin(em), 


otherwise the in-degree of V would be at least k + 1. 

Now by the cyclic permutation of P it follows that any arc e not in the walk 
has neither initial nor final vertex in common with any arc in the walk. So if 
P is not an Eulerian trail, G is not connected. | 


There is an important connection between Eulerian trails and oriented trees: 


Lemma E. Let (e1,...,€m) be an Eulerian trail of a directed graph G having 
no isolated vertices. Let R = fin(em) = init(e1). For each vertex V# R let e[V] 
be the last exit from V in the trail; that is, 


e[V] = ej if init(e;) =V and init(e,) AV forj<k<m. (1) 
Then the vertices of G with the arcs e[V] form an oriented tree with root R. 


Proof. Properties (a) and (b) of the definition of oriented tree are evidently 
satisfied. By exercise 7 we need only show that there are no oriented cycles 
among the e[V]; but this is immediate, since if fin(e[V]) = V’ = init(e[V’]), 
where e[V] = ej and e[V’] = ej, then j <j’. J 


This lemma can perhaps be better understood if we turn things around 
and consider the “first entrances” to each vertex; the first entrances form an 
unordered tree with all arcs pointing away from R. Lemma E has a surprising and 
important converse, proved by T. van Aardenne-Ehrenfest and N. G. de Bruijn 
[Simon Stevin 28 (1951), 203-217]: 
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Theorem D. Let G be a finite, balanced, directed graph, and let G” be an 
oriented tree consisting of the vertices of G plus some of the arcs of G. Let R 
be the root of G’ and let e[V] be the arc of G’ with initial vertex V. Let e, be 
any arc of G with init(e;) = R. Then P = (e1,€2,...,@m) is an Eulerian trail if 
it is an oriented walk for which 


i) no arc is used more than once; that is, ej # ex when j # k. 

ii) e[V] is not used in P unless it is the only choice consistent with rule (i); 
that is, if ej = e[V] and if e is an arc with init(e) = V, then e = ex for some 
k<j. 

iii) P terminates only when it cannot be continued by rule (i); that is, if 
init(e) = fin(e,,), then e = ep for some k. 


Proof. By (iii) and the argument in the proof of Theorem G, we must have 
fin(em) = init(e1) = R. Now if e is an arc not appearing in P, let V = fin(e). 
Since G is balanced, it follows that V is the initial vertex of some arc not in P; 
and if V Æ R, condition (ii) tells us that e[V] is not in P. Now use the same 
argument with e = e[V], and we ultimately find that R is the initial vertex of 
some arc not in the walk, contradicting (iii). 4 


The essence of Theorem D is that it shows us a simple way to construct an 
Eulerian trail in a balanced directed graph, given any oriented subtree of the 
graph. (See the example in exercise 14.) In fact, Theorem D allows us to count 
the exact number of Eulerian trails in a directed graph; this result and many 
other important consequences of the ideas developed in this section appear in 
the exercises that follow. 


EXERCISES 


1. [M20] Prove that if V and V’ are vertices of a directed graph and if there is an 
oriented walk from V to V’, then there is a simple oriented path from V to V”. 


2. [15] Which of the ten “fundamental cycles” listed in (3) of Section 2.3.4.1 are 
oriented cycles in the directed graph (Fig. 32) of that section? 


3. [16] Draw the diagram for a directed graph that is connected but not rooted. 


> 4. [M20] The concept of topological sorting can be defined for any finite directed 
graph G as a linear arrangement of the vertices ViV2...Vn such that init(e) precedes 
fin(e) in the ordering for all arcs e of G. (See Section 2.2.3, Figs. 6 and 7.) Not all finite 
directed graphs can be topologically sorted; which ones can be? (Use the terminology 
of this section to give the answer.) 


5. [M16] Let G be a directed graph that contains an oriented walk (e1,..., en) with 
fin(en) = init(e1). Give a proof that G is not an oriented tree, using the terminology 
defined in this section. 


6. [M21] True or false: A directed graph that is rooted and contains no cycles and 
no oriented cycles is an oriented tree. 


> 7. [M22] True or false: A directed graph satisfying properties (a) and (b) of the 
definition of oriented tree, and having no oriented cycles, is an oriented tree. 
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8. [HM40] Study the properties of automorphism groups of oriented trees, namely 
the groups consisting of all permutations 7 of the vertices and arcs for which we have 
init(ew) = init(e)z, fin(er) = fin(e)z. 


9. [18] By assigning directions to the edges, draw the oriented tree corresponding to 
the free tree in Fig. 30 on page 363, with G as the root. 


10. [22] An oriented tree with vertices Vi,..., Va can be represented inside a computer 
by using a table P[1],..., P[n] as follows: If V; is the root, P[j] = 0; otherwise P[j] = k, 
if the arc e[V;] goes from V; to Ve. (Thus P[1],..., P[n] is the same as the “parent” 
table used in Algorithm 2.3.3E.) 

The text shows how a free tree can be converted into an oriented tree by choosing 
any desired vertex to be the root. Consequently, it is possible to start with an 
oriented tree that has root R, then to convert this into a free tree by neglecting the 
orientation of the arcs, and finally to assign new orientations, obtaining an oriented 
tree with any specified vertex as the root. Design an algorithm that performs this 
transformation: Starting with a table P[1],..., P[n], representing an oriented tree, and 
given an integer j, 1 < j < n, design the algorithm to transform the P table so that it 
represents the same free tree but with V; as the root. 


11. [28] Using the assumptions of exercise 2.3.4.1-6, but with (ax, bx) representing 
an arc from Va, to Vj,, design an algorithm that not only prints out a free subtree as 
in that algorithm, but also prints out the fundamental cycles. [Hint: The algorithm 
given in the solution to exercise 2.3.4.1-6 can be combined with the algorithm in the 
preceding exercise. ] 


12. [M10] In the correspondence between oriented trees as defined here and oriented 
trees as defined at the beginning of Section 2.3, is the degree of a tree node equal to 
the in-degree or the out-degree of the corresponding vertex? 


13. [M24] Prove that if R is a root of a (possibly infinite) directed graph G, then 
G contains an oriented subtree with the same vertices as G and with root R. (Asa 
consequence, it is always possible to choose the free subtree in flow charts like Fig. 32 
of Section 2.3.4.1 so that it is actually an oriented subtree; this would be the case in 
that diagram if we had selected e13, e19, €20, and e17 instead of e13, €19, €23, and e15.) 


14. [21] Let G be the balanced digraph shown in Fig. 36, and let G” be the oriented 
subtree with vertices Vo, Vi, V2 and arcs eọ1, e21. Find all oriented walks P that meet 
the conditions of Theorem D, starting with arc e12. 


15. [M20] True or false: A directed graph that is connected and balanced is strongly 
connected. 


16. [M24] Ina popular solitaire game called “clock,” the 52 cards of an ordinary deck 
of playing cards are dealt face down into 13 piles of four each; 12 piles are arranged 
in a circle like the 12 hours of a clock and the thirteenth pile goes in the center. The 
solitaire game now proceeds by turning up the top card of the center pile, and then 
if its face value is k, by placing it next to the kth pile. (The numbers 1,2,...,13 are 
equivalent to A,2,...,10,J,Q,K.) Play continues by turning up the top card of the 
kth pile and putting it next to its pile, etc., until we reach a point where we cannot 
continue since there are no more cards to turn up on the designated pile. (The player 
has no choice in the game, since the rules completely specify what to do.) The game is 
won if all cards are face up when play terminates. [Reference: E. D. Cheney, Patience 
(Boston: Lee & Shepard, 1870), 62-65; the game was called “Travellers’ Patience” in 
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England, according to M. Whitmore Jones, Games of Patience (London: L. Upcott 
Gill, 1900), Chapter 7.] 

Show that the game will be won if and only if the following directed graph is an 
oriented tree: The vertices are Vi, V2,..., Via; the arcs are e1, €2,...,e12, where ej goes 
from V; to Vz if k is the bottom card in pile j after the deal. 

In particular, if the bottom card of pile j is a “j”, for j # 13, it is easy to see 
that the game is certainly lost, since this card could never be turned up. The result 
proved in this exercise gives a much faster way to play the game!) 


17. [M32] What is the probability of winning the solitaire game of clock (described 
in exercise 16), assuming the deck is randomly shuffled? What is the probability that 
exactly k cards are still face down when the game is over? 


18. [M30] Let G be a graph with n+1 vertices Vo, Vi,..., Vn and m edges €1,...,€m. 
Make G into a directed graph by assigning an arbitrary orientation to each edge; then 
construct the m x (n+ 1) matrix A with 


+1, if init(e;) = V;; 
aij = —1, if fin(e:) = Vj; 
0, otherwise. 


Let Ao be the m x n matrix A with column 0 deleted. 


a) If m =n, show that the determinant of Ao is equal to 0 if G is not a free tree, and 
equal to +1 if G is a free tree. 

b) Show that for general m the determinant of Aj Ao is the number of free subtrees 
of G (namely the number of ways to choose n of the m edges so that the resulting 
graph is a free tree). [Hint: Use (a) and the result of exercise 1.2.3-46.] 


19. [M31] (The matrix tree theorem.) Let G be a directed graph with n + 1 vertices 
Vo, Vi, ---; Vn. Let A be the (n+ 1) x (n+ 1) matrix with 


aoe —k, ift# J and there are k arcs from V; to Vj; 
a t, ifi= and there are t arcs from V; to other vertices. 


(It follows that aio + aii +--+: + Gin = 0 for 0 < i < n.) Let Ao be the same matrix 
with row 0 and column 0 deleted. For example, if G is the directed graph of Fig. 36, 
we have 
2 —2 0 
A=|[-1 3 2], Ay=(_2 E. 
—1 -1 2 


a) Show that if aoo = 0 and aj; = 1 for 1 < j < n, and if G contains no arcs from a 
vertex to itself, then det Ao = [G is an oriented tree with root Vo]. 

b) Show that in general, det Ao is the number of oriented subtrees of G rooted at Vo 
(namely the number of ways to select n of the arcs of G so that the resulting 
directed graph is an oriented tree, with Vo as the root). [Hint: Use induction on 
the number of arcs.] 


20. [M21] If G is an undirected graph on n+1 vertices Vo,..., Vn, let B be then xn 
matrix defined as follows for 1 < i,j < n: 


t, ifi= j and there are t edges touching Vj; 
bij = 4 —1, ift AJ and V; is adjacent to Vj; 
0, otherwise. 
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For example, if G is the graph of Fig. 29 on page 363, with (Vo, Vi, V2, V3, Va) = 
(A, B,C, D, E), we find that 


3 0 -1 -1 
0 2 —1 0 
—1 -1 3-1 
—1 0 -1 2 


B= 


Show that the number of free subtrees of G is det B. [Hint: Use exercise 18 or 19.] 


21. [HM38] (T. van Aardenne-Ehrenfest and N. G. de Bruijn.) Figure 36 is an 
example of a directed graph that is not only balanced, it is regular, which means that 
every vertex has the same in-degree and out-degree as every other vertex. Let G be 
a regular digraph with n vertices Vo, Vi,..., Vn—1, in which every vertex has in-degree 
and out-degree equal to m. (Hence there are mn arcs in all.) Let G* be the digraph 
with mn vertices corresponding to the arcs of G; let a vertex of G* corresponding to 
an arc from V; to Vp in G be denoted by Vjx. An arc goes from Vj, to Vjrgs in G* if 
and only if k = j’. For example, if G is the directed graph of Fig. 36, G* is shown in 
Fig. 37. An Eulerian trail in G is a Hamiltonian cycle in G* and conversely. 

Prove that the number of oriented subtrees of G* is m” Ð)” times the number of 
oriented subtrees of G. [Hint: Use exercise 19.] 


Fig. 37. Arc digraph corresponding to Fig. 36. (See exercise 21.) 
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> 22. [M26] Let G be a balanced, directed graph with vertices Vi, V2,..., Vn and no 
isolated vertices. Let o; be the out-degree of Vj. Show that the number of Eulerian 


trails of G is 


(o +o2+-+-+0n)T [[ (0; —1)!, 


j=l 


where T is the number of oriented subtrees of G with root Vi. [Note: The factor 
(91 +---+ on), which is the number of arcs of G, may be omitted if the Eulerian trail 
(€1,...,;@m) is regarded as equal to (e€x,...,@m;€1,---;€k—1)-] 


> 23. [M33] (N. G. de Bruijn.) For each sequence of nonnegative integers x1,...,@% 
less than m, let f(v1,...,2,%) be a nonnegative integer less than m. Define an infinite 
sequence as follows: Xı = X2 <- = Xp = 0; Xntegi = f(Xnte,.-.,Xn41) when 
n > 0. For how many of the m™ possible functions f is this sequence periodic 
with a period of the maximum length m*? [Hint: Construct a directed graph with 
vertices (%1,...,Uk-1) for all 0 < x; < m, and with arcs from (#1, £2,...,£k—1) to 
(£2,...,£k—1, Zk); apply exercises 21 and 22.] 


> 24. [M20] Let G be a connected digraph with arcs eo, €1,..., €m. Let Eo, Fi,..., Em 
be a set of positive integers that satisfy Kirchhoff’s law for G; that is, for each vertex V, 


Y B= SY E 


init(e;)=V fin(e; )=V 


Assume further that Eo = 1. Prove that there is an oriented walk in G from fin(eo) to 
init(eo) such that edge ej appears exactly Æj times, for 1 < j < m, while edge eo does 
not appear. [Hint: Apply Theorem G to a suitable directed graph.] 


25. [26] Design a computer representation for directed graphs that generalizes the 
right-threaded binary tree representation of a tree. Use two link fields ALINK, BLINK 
and two one-bit fields ATAG, BTAG; and design the representation so that: (i) there is 
one node for each arc of the directed graph (not for each vertex); (ii) if the directed 
graph is an oriented tree with root R, and if we add an arc from R to a new vertex H, 
then the representation of this directed graph is essentially the same as a right-threaded 
representation of this oriented tree (with some order imposed on the children in each 
family), in the sense that ALINK, BLINK, BTAG are respectively the same as LLINK, 
RLINK, RTAG in Section 2.3.2; and (iii) the representation is symmetric in the sense that 
interchanging ALINK, ATAG, with BLINK, BTAG is equivalent to changing the direction on 
all the arcs of the directed graph. 


> 26. [HM39] (Analysis of a random algorithm.) Let G be a directed graph on the 
vertices Vi, V2,...,Vn. Assume that G represents the flow chart for an algorithm, 
where V; is the Start vertex and V, is the Stop vertex. (Therefore V, is a root of G.) 
Suppose each arc e of G has been assigned a probability p(e), where the probabilities 
satisfy the conditions 


0 < ple) <1; 5 ple) =1 forl<j<n. 
init(e)=Vj 


Consider a random walk, which starts at Vı and subsequently chooses branch e of G 
with probability p(e), until V, is reached; the choice of branch taken at each step is to 
be independent of all previous choices. 
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For ae se the graph of exercise 2.3.4.1-7, and assign the respective 


probabilities 1, 4 on z, z, l; 3, +, 4 T + to arcs €1,€2,...,€9. Then the walk “Start—A-— 
” 1 1.1.3 1_ 3 
B-C-A-D-B-C-Stop” is chosen with probability 1-5 -1-5-5- 7-1-7 = 8 


Such random walks are called Markov chains, after the Russian mathematician 
Andrei A. Markov, who first made extensive studies of stochastic processes of this 
kind. The situation serves as a model for certain algorithms, although our requirement 
that each choice must be independent of the others is a very strong assumption. The 
purpose of this exercise is to analyze the computation time for algorithms of this kind. 

The analysis is facilitated by considering the n x n matrix A = (aij), where 
aij = >> ple) summed over all arcs e that go from V; to Vj. If there is no such arc, 
aij = 0. The matrix A for the example considered above is 


010000 
00 40 4 0 
00010 0 
04004 4 
00200 4 
000000 


It follows easily that (A*);; is the probability that a walk starting at V; will be at V; 
after k steps. 
Prove the following facts, for an arbitrary directed graph G of the stated type: 
a) The matrix (I — A) is nonsingular. [Hint: Show that there is no nonzero vector x 
with cA” = zx.] 
b) The average number of times that vertex V; appears in the walk is 


(I - A); = cofactorjı (I — A)/det(I — A), forl<j<n. 


[Thus in the example conreen we find that the vertices A, B, C, D are traversed 
respectively + 43 5 z a3 : times, on the average.] 
c) The probability that V; occurs in the walk is 


a; = cofactor;1(I — A)/cofactor;;(I — A); 


furthermore, an = 1, so the walk terminates in a finite number of steps with 
probability one. 
d) The probability that a random walk starting at V; will never return to V; is 
bj = det (I — A)/cofactor;;(I — A). 
e) The probability that V; occurs exactly k times in the walk is a;(1 — b;)*~*b,, for 
k>1l<j<n. 
27. [M30] (Steady states.) Let G be a directed graph on vertices Vi, ..., Vn, whose 
arcs have been assigned probabilities p(e) as in exercise 26. Instead of having Start 
and Stop vertices, however, assume that G is strongly connected; thus each ver- 
tex V; is a root, and we assume that the probabilities p(e) are positive and satisfy 
Z init(e)=v; p(e) = 1 for all j. A random process of the kind described in exercise 26 is 


said to have a “steady state” (21,...,@n) if 
t= 5 ple) £i, forl<j<n. 
init(e)=V; 


fin(e)=Vj 
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Let tj be the sum, over all oriented subtrees Tj of G that are rooted at Vj, of the 
products [[ er, p(e). Prove that (t,...,tn) is a steady state of the random process. 
J 


> 28. [M35] Consider the (m+n) x (m+n) determinant illustrated here for m = 2 and 
n=3: 


aotaitaz2t+tai3 0 aii ai2 ai3 
0 azo +a21 + a22 + a23 a1 a22 a23 
det bi bi2 biotbi1+bi2 0 0 
b21 b22 0O b29 + b21+ 622 0 
b31 b32 0 0 b390 + b31 + b32 


Show that when this determinant is expanded as a polynomial in the a’s and b’s, each 
nonzero term has coefficient +1. How many terms appear in the expansion? Give a 
rule, related to oriented trees, that characterizes exactly which terms are present. 


*2.3.4.3. The “infinity lemma.” Until now we have concentrated mainly on 
trees that have only finitely many vertices (nodes), but the definitions we have 
given for free trees and oriented trees apply to infinite graphs as well. Infinite 
ordered trees can be defined in several ways; we can, for example, extend the 
concept of “Dewey decimal notation” to infinite collections of numbers, as in 
exercise 2.3-14. Even in the study of computer algorithms there is occasionally 
a need to know the properties of infinite trees — for example, to prove by contra- 
diction that a certain tree is not infinite. One of the most fundamental properties 
of infinite trees, first stated in its full generality by D. Konig, is the following: 


Theorem K (The “infinity lemma”). Every infinite oriented tree in which 
every vertex has finite degree has an infinite path to the root, that is, an infinite 
sequence of vertices Vo, Vi, V2, ... in which Vo is the root and fin (e[V;j+1]) = V; 
for all j > 0. 


Proof. We define the path by starting with Vo, the root of the oriented tree. 
Assume that j > 0 and that V; has been chosen having infinitely many de- 
scendants. The degree of V; is finite by hypothesis, so V; has finitely many 
children U,,...,U,. At least one of these children must possess infinitely many 
descendants, so we take V;,1 to be such a child of V}. 

Now Vo, Vi, V2, ... is an infinite path to the root. I 


Students of calculus may recognize that the argument used here is essentially 
like that used to prove the classical Bolzano—Weierstrass theorem, “A bounded 
infinite set of real numbers has an accumulation point.” One way of stating 
Theorem K, as Konig observed, is this: “If the human race never dies out, 
somebody now living has a line of descendants that will never die out.” 

Most people think that Theorem K is completely obvious when they first 
encounter it, but after more thought and a consideration of further examples 
they realize that there is something profound about it. Although the degree of 
each node of the tree is finite, we have not assumed that the degrees are bounded 
(less than some number N for all vertices), so there may be nodes with higher 
and higher degrees. It is at least conceivable that everyone’s descendants will 
ultimately die out although there will be some families that go on a million 
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generations, others a billion, and so on. In fact, H. W. Watson once published a 
“proof” that under certain laws of biological probability carried out indefinitely, 
there will be infinitely many people born in the future but each family line will 
die out with probability one. His paper |J. Anthropological Inst. Gt. Britain and 
Ireland 4 (1874), 138-144] contains important and far-reaching theorems in spite 
of the minor slip that caused him to make this statement, and it is significant 
that he did not find his conclusions to be logically inconsistent. 

The contrapositive of Theorem K is directly applicable to computer algo- 
rithms: If we have an algorithm that periodically divides itself up into finitely 
many subalgorithms, and if each chain of subalgorithms ultimately terminates, 
then the algorithm itself terminates. 

Phrased yet another way, suppose we have a set S, finite or infinite, such 
that each element of S is a sequence (21, £2, ..., Zn) of positive integers of finite 
length n > 0. If we impose the conditions that 


i) if (£1,..., £n) is in S, so is (£1,..., £p) for 0 < k <n; 
ii) if (£1,...,£n) is in S, only finitely many zn+1 exist for which (z1,..., 
Zn, Zn+1) is also in S; 
iii) there is no infinite sequence (x1, £2,...) all of whose initial subsequences 
(£1, £2,..., £n) lie in S; 


then S is essentially an oriented tree, specified essentially in a Dewey decimal 
notation, and Theorem K tells us that S is finite. 

One of the most convincing examples of the potency of Theorem K arises in 
connection with a family of interesting tiling problems introduced by Hao Wang. 
A tetrad type is a square divided into four parts, each part having a specified 


number in it, such as 

ZN > 
The problem of tiling the plane is to take a finite set of tetrad types, with an 
infinite supply of tetrads of each type, and to show how to place one in each 
square of an infinite plane (without rotating or reflecting the tetrad types) in 
such a way that two tetrads are adjacent only if they have equal numbers where 
they touch. For example, we can tile the plane using the six tetrad types 


2 12 22 1 11 21 


in essentially only one way, by repeating the rectangle 


1 X 21 
—— (3) 
1 X 21 
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over and over. The reader may easily verify that there is no way to tile the plane 
with the three tetrad types 


1 1 2 
3X1 1X2 23X3 |: (4) 
2 1 2 


Wang’s observation [Scientific American 213, 5 (November 1965), 98-106] is 
that if it is possible to tile the upper right quadrant of the plane, it is possible to 
tile the whole plane. This is certainly unexpected, because a method for tiling 
the upper right quadrant involves a “boundary” along the x and y axes, and it 
would seem to give no hint as to how to tile the upper left quadrant of the plane 
(since tetrad types may not be rotated or reflected). We cannot get rid of the 
boundary merely by shifting the upper-quadrant solution down and to the left, 
since it does not make sense to shift the solution by more than a finite amount. 
But Wang’s proof runs as follows: The existence of an upper-right-quadrant 
solution implies that there is a way to tile a 2n x 2n square, for all n. The set 
of all solutions to the problem of tiling squares with an even number of cells on 
each side forms an oriented tree, if the children of each 2n x 2n solution x are 
the possible (2n + 2) x (2n + 2) solutions that can be obtained by bordering z. 
The root of this oriented tree is the 0 x 0 solution; its children are the 2 x 2 
solutions, etc. Each node has only finitely many children, since the problem of 
tiling the plane assumes that only finitely many tetrad types are given; hence by 
the infinity lemma there is an infinite path to the root. This means that there 
is a way to tile the whole plane (although we may be at a loss to find it)! 

For later developments in tetrad tiling, see the beautiful book Tilings and 
Patterns by B. Griinbaum and G. C. Shephard (Freeman, 1987), Chapter 11. 


EXERCISES 


1. [M10] The text refers to a set S containing finite sequences of positive integers, 
and states that this set is “essentially an oriented tree.” What is the root of this 
oriented tree, and what are the arcs? 


2. [20] Show that if rotation of tetrad types is allowed, it is always possible to tile 
the plane. 


3. [M23] If it is possible to tile the upper right quadrant of the plane when given an 
infinite set of tetrad types, is it always possible to tile the whole plane? 


4. [M25] (H. Wang.) The six tetrad types (2) lead to a toroidal solution to the 
tiling problem, that is, a solution in which some rectangular pattern — namely (3) — 
is replicated throughout the entire plane. 

Assume without proof that whenever it is possible to tile the plane with a finite 
set of tetrad types, there is a toroidal solution using those tetrad types. Use this 
assumption together with the infinity lemma to design an algorithm that, given the 
specifications of any finite set of tetrad types, determines in a finite number of steps 
whether or not there exists a way to tile the plane with these types. 


5. [M40] Show that using the following 92 tetrad types it is possible to tile the plane, 
but that there is no toroidal solution in the sense of exercise 4. 
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To simplify the specification of the 92 types, let us first introduce some notation. 
Define the following “basic codes”: 


a=(1,2,1,2) 6=(3,4,2,1) »y=(2, 1,3, 4) 6 =(4, 3,4, 3) 
a =(Q,D, P, R) b=( , ,L,P) c =(U,Q,T, S) d=(, ,S,T) 
N=(Y, ,X, ) J=(D,U, ,X) K=(,Y,R,L) B=(,,, ) 
R=(, RR) L=(, LD) P=(, ,PP) S=(, ,S,S) 
T=(,,7,T) X=(, ,X,X) 
Y=(Y,Y, ’ ) U=(U,U, ) ) D=(D,D, , ) Q=(Q,Q, , ) 
The tetrad types are now 
a{a, b,c, d} 4 types] 
B{Y{B,U, QHP, T}, {B,U, D,QHP, S, T}, K{B,U, Q}} 21 types] 
V{{{X, BHL, P, S,T}, RHB, Q}, J{L, P, S,T}} 22 types] 
O{X{L, P,S,T}{B, Q}, Y{B,U, QH{P,T}, N{a, b,c, a}, 
J{L,P,S,T}, K{B,U,Q}, {R, L, P,S,T}{B,U,D,Q}} [45 types] 


These abbreviations mean that the basic codes are to be put together component by 
component and sorted into alphabetic order in each component; thus 


BY{B,U, QHP, T} 
stands for six types BYBP, BYUP, GBYQP, GYBT, BYUT, BYQT. The type BYQT is 
(3,4,2,1)(Y,Y, , )(Q2,Q, , )(, .T,T) = (3QY, 4QY, 27, 1T) 


after multiplying corresponding components and sorting into order. 
This is intended to correspond to the tetrad type shown on the right, 3QY, 
where we use strings of symbols instead of numbers in the four quarters 2TX1T 
of the type. Two tetrad types can be placed next to each other only if AQY 
they have the same string of symbols at the place they touch. and 
A $-tetrad is one that has a £ in its specification as given above. To get started 
on the solution to this exercise, note that any 6-tetrad must have an a-tetrad to its 
left and to its right, and a 6-tetrad above and below. An aa-tetrad must have 8K B or 
BKU or EKQ to its right, and then must come an ab-tetrad, etc. 
(This construction is a simplified version of a similar one given by Robert Berger, 
who went on to prove that the general problem in exercise 4, without the invalid 
assumption, cannot be solved. See Memoirs Amer. Math. Soc. 66 (1966).) 


6. [M23] (Otto Schreier.) In a famous paper [Nieuw Archief voor Wiskunde (2) 15 
(1927), 212-216], B. L. van der Waerden proved the following theorem: 
Ifk and m are positive integers, and if we have k sets S1,..., Sp of positive integers 
with every positive integer included in at least one of these sets, then at least one 
of the sets S; contains an arithmetic progression of length m. 


(The latter statement means there exist integers a and ô > 0 such that a + 6, a+ 26, 
.., a+ mô are all in S;.) If possible, use this result and the infinity lemma to prove 
the following stronger statement: 


If k and m are positive integers, there is a number N such that if we have k sets 
S1,..., Sp of integers with every integer between 1 and N included in at least one 
of these sets, then at least one of the sets S; contains an arithmetic progression of 
length m. 
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> 7. [M30] If possible, use van der Waerden’s theorem of exercise 6 and the infinity 
lemma to prove the following stronger statement: 
If k is a positive integer, and if we have k sets S1,...,S% of integers with every 
positive integer included in at least one of these sets, then at least one of the 
sets S; contains an infinitely long arithmetic progression. 
> 8. [M39] (J. B. Kruskal.) If T and T’ are (finite, ordered) trees, let the notation 
T C T' signify that T can be embedded in T’, as in exercise 2.3.2-22. Prove that if 
Tı, T2, T3, ... is any infinite sequence of trees, there exist integers j < k such that 
T; C Tp. (In other words, it is impossible to construct an infinite sequence of trees in 
which no tree contains any of the earlier trees of the sequence. This fact can be used 
to prove that certain algorithms must terminate.) 


*2.3.4.4. Enumeration of trees. Some of the most instructive applications of 
the mathematical theory of trees to the analysis of algorithms are connected with 
formulas for counting how many different trees there are of various kinds. For 
example, if we want to know how many different oriented trees can be constructed 
having four indistinguishable vertices, we find that there are just 4 possibilities: 


A J. (a) 


For our first enumeration problem, let us determine the number a, of 
structurally different oriented trees with n vertices. Obviously, a, = 1. Ifn > 1, 
the tree has a root and various subtrees; suppose there are jı subtrees with 1 
vertex, j2 with 2 vertices, etc. Then we may choose jp of the ap possible k-vertex 


trees in 
+ jk - ") 
Jk 


ways, since repetitions are allowed (exercise 1.2.6-60), and so we see that 
i — 1 = In-1 — 1 
An = 5y ~ Je” 1+ Jn 1 yi esi @ 
ji t2jo+--=n-1 Jı Jn—1 


If we consider the generating function A(z) = 5>,,anz”, with ao = 0, we find 


that the identity 
1 E a+j-—1 F 
dary 3 ( j ) Í 


together with (2) implies 
z 


L (1 = z) (1 -— z?)e(1-— z3)... k 


This is not an especially nice form for A(z), since it involves an infinite product 
and the coefficients a1, a2,... appear on the right-hand side. A somewhat more 
aesthetic way to represent A(z) is given in exercise 1; it leads to a reasonably 
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efficient formula for calculating the values an (see exercise 2) and, in fact, it also 
can be used to deduce the asymptotic behavior of a,, for large n (see exercise 4). 
We find that 


A(z) = z + 2? + 223 + 424 + 925 + 2028 + 4827 + 11528 
+ 28629 + 719279 + 18422" +--+. (4) 


Now that we have essentially found the number of oriented trees, it is quite 
interesting to determine the number of structurally different free trees with n 
vertices. There are just two distinct free trees with four vertices, namely 


——+— ad | (5) 


because the first two and last two oriented trees of (1) become identical when 
the orientation is dropped. 

We have seen that it is possible to select any vertex X of a free tree and to 
assign directions to the edges in a unique way so that it becomes an oriented tree 
with X as root. Once this has been done, for a given vertex X, suppose there are 
k subtrees of the root X, with s1, s2,...,s, vertices in these respective subtrees. 
Clearly, k is the number of arcs touching X, and sı + s9+---+s,=n—-—1. In 
these circumstances we say that the weight of X is max(s1, s2, ..., Sk). Thus in 
the tree 


C D G (6) 


J K H 
the vertex D has weight 3 (each of the subtrees leading from D has three of the 
nine remaining vertices), and vertex E has weight max(7,2) = 7. A vertex with 
minimum weight is called a centroid of the free tree. 

Let X and 51, 52,...,5% be as above, and let Y1, Y2,..., Ypk be the roots of 
the subtrees emanating from X. If Y is any node in the Yı subtree, its weight 
must be at least n— sı = 1+s2+---+8,, since when Y is the assumed root there 
are at least n — sı vertices in its subtree containing X. Thus if Y is a centroid 
we have 


weight (X) = max (s1, 52,...,8%) > weight (Y) > 1+ s2+---+ Sk, 


and this is possible only if sı > s2 +---+ sk. A similar result may be derived if 
we replace Yı by Y; in this discussion. So at most one of the subtrees at a vertex 
can contain a centroid. 
This is a strong condition, for it implies that there are at most two centroids 
in a free tree, and if two centroids exist, they are adjacent. (See exercise 9.) 
Conversely, if s1 > S2 +--+ sk, there is a centroid in the Y; subtree, since 


weight (Y1) < max (sı — 1, 1 + s2 ++- + sk) < sı = weight (X), 
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and the weight of all nodes in the Y2,..., Yp subtrees is at least sı +1. We have 
proved that the vertex X is the only centroid of a free tree if and only if 


8) S s1 +s + SE Sj; forl<j<k. (7) 


Therefore the number of free trees with n vertices, having only one centroid, 
is the number of oriented trees with n vertices minus the number of such oriented 
trees violating condition (7); the latter consist essentially of an oriented tree with 
sj vertices and another oriented tree with n— sj < sj vertices. The number with 
one centroid therefore comes to 


Gn — A1An—1 — 424n—2 — *** — A|n/2|4fn/2)- (8) 


A free tree with two centroids has an even number of vertices, and the weight of 
each centroid is n/2 (see exercise 10). So if n = 2m, the number of bicentroidal 
free trees is the number of choices of 2 things out of a,, with repetition, namely 


(2) 

5 ; 
To get the total number of free trees, we therefore add $an/2(@n/2 + 1) to (8) 
when n is even. The form of Eq. (8) suggests a simple generating function, and 


indeed, we find without difficulty that the generating function for the number of 
structurally different free trees is 


F(z) = A(z) - TO + LAG?) 


= z +2? + 23 +224 +325 + 6z + 112z" + 2328 
+ 472° +1062"? +2352! +---. (9) 


This simple relation between F(z) and A(z) is due primarily to C. Jordan, who 
considered the problem in 1869. 

Now let us turn to the question of enumerating ordered trees, which are our 
principal concern with respect to computer programming algorithms. There are 
five structurally different ordered trees with four vertices: 


NALA « 


The first two are identical as oriented trees, so only one of them appeared in (1) 
above. 

Before we examine the number of different ordered tree structures, let us 
first consider the case of binary trees, since this is closer to the actual computer 
representation and it is easier to study. Let bn be the number of different binary 
trees with n nodes. From the definition of binary tree it is apparent that bo = 1, 
and for n > 0 the number of possibilities is the number of ways to put a binary 
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tree with k nodes to the left of the root and another with n — 1 — k nodes to the 
right. So 
bn = bobn—1 + bibn—2 +--+ + bn—1b0, n > 1. (11) 


From this relation it is clear that the generating function 
B(z) = bo + bız + bgz? +- 


satisfies the equation 
2B(z)? = B(z)—1. (12) 


Solving this quadratic equation and using the fact that B(0) = 1, we obtain 


Ble) = + (1- vI-#2) = z(0-¥ ($) caa") 


VA 
k>0 
1 1 
= ne —4z)” 
e o EC) SR 
n>0 m n>0 G M 
_ 2N git 
Ee njn+l1 
=1 +z +22? +5294 1424 4+ 422° + 13226 + 42927 


+ 143028 + 48622? + 167962 1° +---. (13) 


(See exercise 1.2.6-47.) The desired answer is therefore 


1 2n 
bn = . 
EE (r) (14) 


By Stirling’s formula, this is asymptotically 4"/n/an + O(4”n75/2). Some 
important generalizations of Eq. (14) appear in exercises 11 and 32. 

Returning to our question about ordered trees with n nodes, we can see that 
this is essentially the same question as the number of binary trees, since we have 
a natural correspondence between binary trees and forests, and a tree minus its 
root is a forest. Hence the number of (ordered) trees with n vertices is bn—1, the 
number of binary trees with n — 1 vertices. 


The enumerations performed above assume that the vertices are indistin- 
guishable points. If we label the vertices 1, 2, 3, 4 in (2) and insist that 1 is to 
be the root, we now get 16 different oriented trees: 


1 1 1 1 il le le le le le 1 
36% 46 $%2@ 4è Q¢@ 8 Be 4¢@ 20 4¢@ 29 3 
4e 3 


4 2 3 2 


1 1 1 
3¢ 46 2d ae 2d 3 /N (15) 
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The question of enumeration for labeled trees is clearly quite different from the 
one solved above. In this case it can be rephrased as follows: “Consider drawing 
three lines, pointing from each of the vertices 2, 3, and 4 to another vertex; 
there are three choices of lines emanating from each vertex, so there are 33 = 27 
possibilities in all. How many of these 27 ways will yield oriented trees with 1 
as the root?” The answer, as we have seen, is 16. A similar reformulation of the 
same problem, this time for the case of n vertices, is the following: “Let f(x) be 
an integer-valued function such that f(1) = 1 and 1 < f(x) < n for all integers 
1<a2 <n. We call f a tree mapping if f(x), that is, f(f(---(f(z))---)) 
iterated n times, equals 1, for all x. How many tree mappings are there?” This 
problem comes up, for example, in connection with random number generation. 
We will find, rather surprisingly, that on the average exactly one out of every n 
such functions f is a tree mapping. 

The solution to this enumeration problem can readily be derived using the 
general formulas for counting subtrees of graphs that have been developed in 
previous sections (see exercise 12). But there is a much more informative way 
to solve the problem, one that gives us a new and compact manner to represent 
oriented tree structure. 

Suppose that we’ve been given an oriented tree with vertices {1,2,...,n} 
and with n — 1 arcs, where the arcs go from j to f(j) for all j except the root. 
There is at least one terminal (leaf) vertex; let Vı be the smallest number of a 
leaf. If n > 1, write down f(Vi) and delete both V; and the arc Vij > f(Vi) 
from the tree; then let V2 be the smallest number whose vertex is terminal in 
the resulting tree. If n > 2, write down f(V2) and delete both V2 and the arc 
V2 > f(V2) from the tree; and proceed in this way until all vertices have been 
deleted except the root. The resulting sequence of n — 1 numbers, 


fV), FV), Keay IVs 1 í FING) < n, (16) 


is called the canonical representation of the original oriented tree. 
For example, the oriented tree 


with 10 vertices has the canonical representation 1, 3, 10, 5, 10, 1, 3, 5, 3. 

The important point here is that we can reverse this process and go from 
any sequence of n — 1 numbers (16) back to the oriented tree that produced it. 
For if we have any sequence x1, £2,...,£n—1 Of numbers between 1 and n, let 
V, be the smallest number that does not appear in the sequence z1,..., En—1; 
then let V2 be the smallest number 4 Vı that does not appear in the sequence 
2,-.-,%,—1; and so on. After obtaining a permutation Vi Və... Vn of the integers 
{1,2,...,n} in this way, draw arcs from vertex Vj to vertex zj, for 1 < j < n. 
This gives a construction of a directed graph with no oriented cycles, and by 
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exercise 2.3.4.2—7 it is an oriented tree. Clearly, the sequence 21, %2,...,%y_1 is 
the same as the sequence (16) for this oriented tree. 

Since the process is reversible, we have obtained a one-to-one correspondence 
between (n — 1)-tuples of numbers {1,2,...,n} and oriented trees on these 
vertices. Hence there are n”~! distinct oriented trees with n labeled vertices. If 
we specify that one vertex is to be the root, there is clearly no difference between 
one vertex and another, so there are n”~? distinct oriented trees on {1,2,...,n} 
having a given root. This accounts for the 16 = 447? trees in (15). From this 
information it is easy to determine the number of free trees with labeled vertices 
(see exercise 22). The number of ordered trees with labeled vertices is also easy to 
determine, once we know the answer to that problem when no labels are involved 
(see exercise 23). So we have essentially solved the problems of enumerating the 
three fundamental classes of trees, with both labeled and unlabeled vertices. 

It is interesting to see what would happen if we were to apply our usual 
method of generating functions to the problem of enumerating labeled oriented 
trees. For this purpose we would probably find it easiest to consider the quantity 
r(n,q), the number of labeled directed graphs with n vertices, with no oriented 
cycles, and with one arc emanating from each of q designated vertices. The 
number of labeled oriented trees with a specified root is therefore r(n, n— 1). In 
this notation we find by simple counting arguments that, for any fixed integer m, 


r(n, q) = (Fronts, k)r(n—m—k,q—k), if0<m<n-gq, (18) 
k 


r(n, d =F (f )rm-1, 4-4), ifg=n-1. (19) 


k 


The first of these relations is obtained if we partition the undesignated vertices 
into two groups A and B, with m vertices in A and n — q — m vertices in B; 
then the q designated vertices are partitioned into k vertices that begin paths 
leading into A, and q—k vertices that begin paths leading into B. Relation (19) 
is obtained by considering oriented trees in which the root has degree k. 

The form of these relations indicates that we can work profitably with the 
generating function 


Gm(z) = r(m, 0) + r(m+ 1, Iz4 


r(m +2, 2)z? | r(k +m, k)z* 
ia? <a sa 
k 


In these terms Eq. (18) says that Gn_—g(z) = Gm(z)Gn—q—m(z), and therefore by 
induction on m, we find that G,,(z) = Gi(z)™. Now from Eq. (19), we obtain 


r(n, n—1)z Swor(n—1,n—1—k)z""! 
Gly= De a as ae 
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In other words, putting G,(z) = w, the solution to our problem comes from 
the coefficients of the solution to the transcendental equation 


w=e™, (20) 


This equation can be solved with the use of Lagrange’s inversion formula: 
z = C/f(¢) implies that 


c= 2 =  g("-0(0), (21) 
n>1 ue 


where gn(¢) = f(¢)”, when f is analytic in the neighborhood of the origin, and 
f(0) 4 0 (see exercise 4.7-16). In this case, we may set ¢ = zw, f(¢) = eS, and 
we deduce the solution 


w= 5 m1) 2”, (22) 


in agreement with the answer obtained above. 

G. N. Raney has shown that we can extend this method in an important 
way to obtain an explicit power series for the solution to the considerably more 
general equation 


w = ye” + ye?” +--+ + ye”, 


solving for w in terms of a power series in y1,...,Ys and 21,...,25. For this 
generalization, let us consider s-dimensional vectors of integers 


n = (n1, Nn2,..., Ns), 
and let us write for convenience 


Yn =ni + no+ e +ns. 


Suppose that we have s colors C1, C2,...,Cs, and consider directed graphs 
in which each vertex is assigned a color; for example, 
e d 6 Yellow 
14Blue (23) 


4e Yellow 5 *°Red 


Let r(n,q) be the number of ways to draw arcs and to assign colors to the 
vertices {1,2,...,n}, such that 
i) for 1 <i < s there are exactly n; vertices of color C; (hence n = Xn); 
ii) there are q arcs, one leading from each of the vertices {1,2,...,q}; 
iii) for 1 <i < s there are exactly qi arcs leading to vertices of color C; (hence 
q = Xa); 


iv) there are no oriented cycles (hence q < n, unless q = n = 0). 


Let us call this an (n,q)-construction. 
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For example, if C1 = red, Co = yellow, and C3 = blue, then (23) shows 
a (352.2); (1, 2, 2))-construction. When there is only one color, we have the 
oriented tree problem that we have already solved. Raney’s idea is to generalize 
the one-dimension construction to s dimensions. 

Let n and q be fixed s-place vectors of nonnegative integers, and let n = Xn, 
q = dq. For each (n,q)-construction and each number k, 1 < k < n, we will 
define a canonical representation consisting of four things: 


a) a number t, with q < t < n; 

b) a sequence of n colors, with n; of color Ci; 

c) a sequence of q colors, with q; of color Ci; 

d) for 1 < i < s, a sequence of q; elements of the set {1,2,...,nj}. 


The canonical representation is defined thus: First list the vertices {1,2,...,q} 
in the order V1, V2,..., V} of the canonical representation of oriented trees (as 
given above), and then write below vertex V; the number f(V;) of the vertex on 
the arc leading from Vj. Let t = f(V,); and let the sequence (c) of colors be the 
respective colors of the vertices f(Vi),..., f(Vq). Let the sequence (b) of colors 
be the respective colors of the vertices k, k+1,...,,1,...,—1. Finally, let 
the ith sequence in (d) be 21, i2,- - , Ziq, Where x;; = m if the jth C;-colored 
element of the sequence f(Vi),...,f(Vq) is the mth C;-colored element of the 
sequence k,k+1,...,n,1,...,k—1. 

For example, consider construction (23) and let k = 3. We start by listing 
Vi,..-,V5 and f(Vi),..., (Vs) below them as follows: 


1 2 4 5 3 
7 6 3 3 6 


Hence t = 6, and sequence (c) represents the respective colors of 7, 6, 3, 3, 6, 
namely red, yellow, blue, blue, yellow. Sequence (b) represents the respective 
colors of 3, 4, 5, 6, 7, 1, 2, namely blue, yellow, red, yellow, red, blue, red. 
Finally, to get the sequences in (d), proceed as follows: 


elements this color elements this color encode column 3 
color in 3,4,5,6,7,1,2 in 7,6,3,3,6 by column 2 
red 5, 7,2 7 2 
yellow 4,6 6,6 2,2 
blue 3,1 3,3 1,1 


Hence the (d) sequences are 2; 2,2; and 1,1. 

From the canonical representation, we can recover both the original (n,q)- 
construction and the number k as follows: From (a) and (c) we know the color of 
vertex t. The last element of the (d) sequence for this color tells us, in conjunction 
with (b), the position of t in the sequence k,...,n,1,...,k — 1; hence we know 
k and the colors of all vertices. Then the subsequences in (d) together with 
(b) and (c) determine f(Vi), f(V2),...,f(V_), and finally the directed graph is 
reconstructed by locating Vj,..., Vq as we did for oriented trees. 
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The reversibility of this canonical representation allows us to count the 
number of possible (n,q)-constructions, since there are n — q choices for (a), 
and the multinomial coefficient 

n 
Nl- -Mns 


( ) 
qis- - -34s 
q2 


choices for (c), and n® n2...m% choices for (d). Dividing by the n choices for k, 
we have the general result 


choices for (b), and 


n-q n! q! 
r(n, q) = n nı! n ! qı! q ! ni ng ee ey ng. (24) 
Lows Tlg? rere e 


Furthermore, we can derive analogs of Eqs. (18) and (19): 
zq : 
raa- Y r(t,k)r(n-t,q-k) if0<m<X5£(n-q), (25) 
k,t uk 
X (t-k)=m 


with the convention that r(0,0) = 1, and r(n,q) = 0 if any n; or qi is negative 
or if q > n; 


raa D D (>91) ra-ena-ke) ifEn=1+54 0 
i=l » 


where e; is the vector with 1 in position į and zeros elsewhere. Relation (25) 
is based on breaking the vertices {q + 1,..., n} into two parts having m and 
n — q — m elements, respectively; the second relation is derived by removing 
the unique root and considering the remaining structure. We now obtain the 
following result: 


Theorem R. [George N. Raney, Canadian J. Math. 16 (1964), 755-762.] Let 


r(n,q) ny nm qı q 
w = y ea) oh ao ee aa 27 
(> )! Yı y 1 ( ) 


D2 (n—-q)=1 


where r(n, q) is defined by (24), and where n, q are s-dimensional integer vectors. 
Then w satisfies the identity 


w= yer” + ye?” +--+ ye”. (28) 
Proof. By (25) and induction on m, we find that 


m r(n, q) nı Ns „qı q 
w = y syi 7 2a wena» 2 
(Zq)! Yı Ys 1 s ( 9) 
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Now by (26), 
= r(n—e;,q—ke;) n a 
w= 5 5 1 yR ee 
i=l k n,q k! (Sq — k)! 
X (n-q)=1 
5 1 k r(n, q) nı n qı 
= 5 Ty Yiži 5 1 Ys Zi 2 
l 
i=l k n,q (Za)! 
2 (n-q)=k 


The special case where s = 1 and z; = 1 in (27) and (28) is especially 
important in applications, so it has become known as the “tree function” 


T(y) = Y yp" = ye™™, (30) 


See Corless, Gonnet, Hare, Jeffrey, and Knuth, Advances in Computational 
Math. 5 (1996), 329-359, for a discussion of this function’s history and some 
of its remarkable properties. 

A survey of enumeration formulas for trees, based on skillful manipulations 
of generating functions, has been given by I. J. Good [Proc. Cambridge Philos. 
Soc. 61 (1965), 499-517; 64 (1968), 489]. More recently, a mathematical theory 
of species developed by André Joyal [Advances in Math. 42 (1981), 1-82] has 
led to a high-level viewpoint in which algebraic operations on generating func- 
tions correspond directly to combinatorial properties of structures. The book 
Combinatorial Species and Tree-like Structures by F. Bergeron, G. Labelle, and 
P. Leroux (Cambridge Univ. Press, 1998), presents numerous examples of this 
beautiful and instructive theory, generalizing many of the formulas derived above. 


EXERCISES 
1. [M20] (G. Pélya.) Show that 
A(z) = z-exp (A(z) + 1 A(z”) + 4 A(z*) 4+... ). 
[Hint: Take logarithms of (3).] 
2. [HM24] (R. Otter.) Show that the numbers an satisfy the following condition: 
NAn+1 = A18n1 + 2a28n2 + +++ + NAnSnn, 


where 


Snk = J Qn+1—jk- 


1<j<n/k 


(These formulas are useful for the calculation of the an, since snk = S(n—k)k + An+1—k-) 
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3. [M40] Write a computer program that determines the number of (unlabeled) free 
trees and of oriented trees with n vertices, for n < 100. (Use the result of exercise 2.) 
Explore arithmetical properties of these numbers; can anything be said about their 
prime factors, or their residues modulo p? 


> 4. [HM39] (G. Pólya, 1937.) Using complex variable theory, determine the asymp- 
totic value of the number of oriented trees as follows: 

a) Show that there is a real number a between 0 and 1 for which A(z) has ra- 
dius of convergence a and A(z) converges absolutely for all complex z such that 
|z| < a, having maximum value A(a) = a < oo. [Hint: When a power series has 
nonnegative coefficients, it either is entire or has a positive real singularity; and 
show that A(z)/z is bounded as z > a—, by using the identity in exercise 1.] 

b) Let 

F(z,w) = exp (zw + A(z”) + 2A(2*) +--+) — w. 
Show that in a neighborhood of (z,w) = (a,a/a), F(z,w) is analytic in each 
variable separately. 

c) Show that at the point (z, w) = (a,a/a), we have OF /Ow = 0; hence a = 1. 

d) At the point (z, w) = (a,1/a) show that 


OF _ k—2 8? F 
a? + ` a an F a 


Oz 
k>2 


e) When |z| = a and z # a, show that OF /Ow # 0; hence A(z) has only one 
singularity on |z| = a. 
f) Prove that there is a region larger than |z| < a in which 
1 1 
A(z) = + — 28 = za) + (1 — z/a)R(2), 


where A(z) is an analytic function of yz — a. 
g) Prove that consequently 


== TV B/27n + O(n e 


[Note: 1/a ~ 2.955765285652, and a\/B/2n ~ 0.439924012571] 


> 5. [M25] (A. Cayley.) Let cn be the number of (unlabeled) oriented trees having 
n leaves (namely, vertices with in-degree zero) and having at least two subtrees at 
every other vertex. Thus c3 = 2, by virtue of the two trees 


A A 


Find a formula analogous to (3) for the generating function 
ż)= 5 inz” 
n 


6. [M25] Let an “oriented binary tree” be an oriented tree in which each vertex has 
in-degree two or less. Find a reasonably simple relation that defines the generating 
function G(z) for the number of distinct oriented binary trees with n vertices, and find 
the first few values. 
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7. [HM40| Obtain asymptotic values for the numbers of exercise 6. (See exercise 4.) 


8. [20] According to Eq. (9), there are six free trees with six vertices. Draw them, 
and indicate their centroids. 


9. [M20] From the fact that at most one subtree of a vertex in a free tree can contain 
a centroid, prove that there are at most two centroids in a free tree; furthermore if there 
are two, then they must be adjacent. 


10. [M22] Prove that a free tree with n vertices and two centroids consists of two free 
trees with n/2 vertices, joined by an edge. Conversely, if two free trees with m vertices 
are joined by an edge, we obtain a free tree with 2m vertices and two centroids. 


11. [M28] The text derives the number of different binary trees with n nodes (14). 
Generalize this to find the number of different t-ary trees with n nodes. (See exercise 
2.3.1-35; a t-ary tree is either empty or consists of a root and t disjoint t-ary trees.) 
Hint: Use Eq. (21) of Section 1.2.9. 


12. [M20] Find the number of labeled oriented trees with n vertices by using deter- 
minants and the result of exercise 2.3.4.2-19. (See also exercise 1.2.3-36.) 


13. [15] What oriented tree on the vertices {1,2,...,10} has the canonical represen- 
tation 3, 1, 4, 1, 5, 9, 2, 6, 5? 

14. [10] True or false: The last entry, f(Vn—1), in the canonical representation of an 
oriented tree is always the root of that tree. 


15. [21] Discuss the relationships that exist (if any) between the topological sort 
algorithm of Section 2.2.3 and the canonical representation of an oriented tree. 


16. [25] Design an algorithm (as efficient as possible) that converts from the canonical 
representation of an oriented tree to a conventional computer representation using 
PARENT links. 


17. [M26] Let f(x) be an integer-valued function, where 1 < f(x) < m for all integers 
1<a<m. Define x = y if f(x) = f(y) for some r,s > 0, where f(x) = x and 
f(a) = f(f'(2)). By using methods of enumeration like those in this section, 
show that the number of functions such that x = y for all x and y is m™~+Q(m), 
where @(m) is the function defined in Section 1.2.11.3. 


18. [24] Show that the following method is another way to define a one-to-one cor- 
respondence between (n — 1)-tuples of numbers from 1 to n and oriented trees with 
n labeled vertices: Let the leaves of the tree be Vi,...,Vx in ascending order. Let 
(Vi, Ve+1, Ve+2,---,Vq) be the path from Vi to the root, and write down the vertices 
Va,+-+) Ve+2; Vk+1. Then let (V2, Vo41, Vo+2,---, Vr) be the shortest oriented path from 
V2 such that V, has already been written down, and write down V,,..., Vg+2, Vq+1- 
Then let (V3, V-+1,..., Vs) be the shortest oriented path from V3 such that V, has 
already been written, and write Vs,...,Vr+1; and so on. For example, the tree (17) 
would be encoded as 3, 1, 3, 3, 5, 10, 5, 10, 1. Show that this process is reversible, 
and in particular, draw the oriented tree with vertices {1,2,...,10} and representation 
3, 1, 4, 1, 5, 9, 2, 6, 5. 

19. [M24] How many different labeled, oriented trees are there having n vertices, k of 


which are leaves (have in-degree zero)? 


20. [M24] (J. Riordan.) How many different labeled, oriented trees are there having 
n vertices, ko of which have in-degree 0, kı have in-degree 1, k2 have in-degree 2, ...? 
(Note that necessarily ko + kı + ko +--+: = n, and ky + 2ko + 3k3 +---=n—1.) 
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> 21. [M21] Enumerate the number of labeled oriented trees in which each vertex has 
in-degree zero or two. (See exercise 20 and exercise 2.3-20.) 


22. [M20] How many labeled free trees are possible with n vertices? (In other words, if 


we are given n vertices, there are 2(2) possible graphs having these vertices, depending 
on which of the (2) possible edges are incorporated into the graph; how many of these 
graphs are free trees?) 

23. [M21] How many ordered trees are possible with n labeled vertices? (Give a 


simple formula involving factorials.) 


24. [M16] All labeled oriented trees with vertices 1, 2, 3, 4 and with root 1 are shown 
in (15). How many would there be if we listed all labeled ordered trees with these 
vertices and this root? 


25. [M20] What is the value of the quantity r(n,q) that appears in Eqs. (18) and 
(19)? (Give an explicit formula; the text only mentions that r(n, n — 1) = n”~?.) 


26. [20] In terms of the notation at the end of this section, draw the ((3, 2, 4), (1, 4, 2))- 
construction, analogous to (23), and find the number k that corresponds to the canon- 
ical representation having t = 8, the sequences of colors “red, yellow, blue, red, yellow, 
blue, red, blue, blue” and “red, yellow, blue, yellow, yellow, blue, yellow”, and the 
index sequences 3; 1, 2, 2, 1; 2, 4. 

> 27. [M28] Let Ui, U2,...,Up,...,Uq; Vi, V2,...,Vr be vertices of a directed graph, 
where 1 < p < q. Let f be any function from the set {p+1,...,q} into the set 
{1,2,...,r}, and let the directed graph contain exactly q — p arcs, from Up to Vex) for 
p< k< q. Show that the number of ways to add r additional arcs, one from each of the 
V’s to one of the U’s, such that the resulting directed graph contains no oriented cycles, 
is q’'p. Prove this by generalizing the canonical representation method; that is, set 
up a one-to-one correspondence between all such ways of adding r further arcs and the 


set of all sequences of integers a1,a2,...,ar, where 1 < a, < q for 1 < k < r, and 
l<a, <p. 

28. [M22] (Bipartite trees.) Use the result of exercise 27 to enumerate the number 
of labeled free trees on vertices Ui, ..., Um, Vi, ..., Vn, such that each edge joins 


U; to Vp for some j and k. 
29. [HM26] Prove that if Ex(r,t) = r(r+kt)*~'/k!, and if zx* = Ina, then 
= 5 Ex(r, t)z" 


k>0 
for fixed t and for sufficiently small |z| and |æ — 1|. [Use the fact that Gm(z) = Gi(z)™ 


in the discussion following Eq. (19).] In this formula, r stands for an arbitrary real 
number. [Note: As a consequence of this formula we have the identity 


>> Er(r, t)En—x(s, t) = E,(r F S, t); 
k=0 


this implies Abel’s binomial theorem, Eq. (16) of Section 1.2.6. Compare also Eq. (30) 
of that section.] 


30. [M23] Let n,2,y,21,...,2n be positive integers. Consider a set of £ + y + zı + 
+++ + Zn +n vertices ri, Sjk, tj (1 <i<xr+y,1<j<n,1< k< zj), in which arcs 
have been drawn from sj, to tj for all j and k. According to exercise 27, there are 
(aty)\(a+tytat-::-4 Za) ways to draw one arc from each of t1,...,tn to other 
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vertices such that the resulting directed graph contains no oriented cycles. Use this 
fact to prove Hurwitz’s generalization of the binomial theorem: 


So r(s+az1+: x bent eet t ay } (1 €1)21 perg (1 edal e 
= (z+y)(z+y+ zi +: + 2n)", 
where the sum is over all 2” choices of €1,...,€n equal to 0 or 1. 


31. [M24] Solve exercise 5 for ordered trees; that is, derive the generating function for 
the number of unlabeled ordered trees with n terminal nodes and no nodes of degree 1. 


32. [M37] (A. Erdélyi and I. M. H. Etherington, Edinburgh Math. Notes 32 (1941), 
7-12.) How many (ordered, unlabeled) trees are there with no nodes of degree 0, nı of 
degree 1, ..., Mm of degree m, and none of degree higher than m? (An explicit solution 
to this problem can be given in terms of factorials, thereby considerably generalizing 
the result of exercise 11.) 


33. [M28] The text gives an explicit power series solution for the equation w = 
yie!” + --- + yre™”, based on enumeration formulas for certain oriented forests. 
Similarly, show that the enumeration formula of exercise 32 leads to an explicit power 
series solution to the equation 


w = zw + zw? + -o H rw, 


expressing w as a power series in 21,...,2r. (Here e1,...,€r are fixed nonnegative 
integers, at least one of which is zero.) 


2.3.4.5. Path length. The concept of the “path length” of a tree is of great 
importance in the analysis of algorithms, since this quantity is often directly 
related to the execution time. Our primary concern is with binary trees, since 
they are so close to actual computer representations. 

In the following discussion we will extend each binary tree diagram by adding 
special nodes wherever a null subtree was present in the original tree, so that 


becomes (1) 


The latter is called an extended binary tree. After the square-shaped nodes have 
been added in this way, the structure is sometimes more convenient to deal with, 
and we shall therefore meet extended binary trees frequently in later chapters. 
It is clear that every circular node has two children and every square node has 
none. (Compare with exercise 2.3-20.) If there are n circular nodes and s square 
nodes, we have n+ s—1 edges (since the diagram is a free tree); counting another 
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way, by the number of children, we see that there are 2n edges. Hence it is clear 
that 


s=n+l,; (2) 


in other words, the number of “external” nodes just added is one more than 
the number of “internal” nodes we had originally. (For another proof, see 
exercise 2.3.1-14.) Formula (2) is correct even when n = 0. 

Assume that a binary tree has been extended in this way. The external path 
length of the tree, E, is defined to be the sum— taken over all external (square) 
nodes — of the lengths of the paths from the root to each node. The internal 
path length, I, is the same quantity summed over the internal (circular) nodes. 
In (1) the external path length is 


H=3+34+24+34+44+44+3+3 = 25, 


and the internal path length is 
$=241+0424+341+4+2=11. 
These two quantities are always related by the formula 


E=1+2n, (3) 


where n is the number of internal nodes. 

To prove formula (3), consider deleting an internal node V at a distance k 
from the root, where both children of V are external. The quantity E goes down 
by 2(k + 1), since the children of V are removed, then it goes up by k, since V 
becomes external; so the net change in FE is —k — 2. The net change in J is —k, 
so (3) follows by induction. 

It is not hard to see that the internal path length (and hence the external 
path length also) is greatest when we have a degenerate tree with linear structure; 
in that case the internal path length is 


(n-1)+(n—-2)+---+14+0= 


It can be shown that the “average” path length over all binary trees is essentially 
proportional to n/n (see exercise 5). 

Consider now the problem of constructing a binary tree with n nodes that 
has minimum path length. Such a tree will be important, since it will minimize 
the computation time for various algorithms. Clearly, only one node (the root) 
can be at zero distance from the root; at most two nodes can be at distance 1 
from the root, at most four can be 2 away, and so on. Therefore the internal 
path length is always at least as big as the sum of the first n terms of the series 


0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4... 
This is the sum X`}; |lgk], which we know from exercise 1.2.4-42 is 
(n+1)g-29! +2, q= |lg(n+1)]. (4) 
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The optimum value (4) is nlgn+O(n), since q = lgn+O(1); it is clearly achieved 
in a tree that looks like this (illustrated for n = 12): 


A tree such as (5) is called the complete binary tree with n internal nodes. 
In the general case we can number the internal nodes 1, 2,...,n; this numbering 
has the useful property that the parent of node k is node |k/2]|, and the children 
of node k are nodes 2k and 2k + 1. The external nodes are numbered n + 1 
through 2n + 1, inclusive. 

It follows that a complete binary tree may simply be represented in sequen- 
tial memory locations, with the structure implicit in the locations of the nodes 
(not in links). The complete binary tree appears explicitly or implicitly in many 
important computer algorithms, so the reader should give it special attention. 

These concepts have important generalizations to ternary, quaternary, and 
higher-order trees. We define a t-ary tree as a set of nodes that is either empty 
or consists of a root and t ordered, disjoint t-ary trees. (This generalizes the 
definition of binary tree in Section 2.3.) Here, for example, is the complete 
ternary tree with 12 internal nodes: 


[26 


It is easy to see that the same construction works for any t > 2. In the complete 
t-ary tree with internal nodes {1,2,...,n}, the parent of node k is node 


[(k +t- 2)/t] = [(k — 1)/t], 
and the children of node k are 


t(k-1)+2, t(k-1)+3, ..., tk+1. 
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This tree has the minimum internal path length among all t-ary trees with n 
internal nodes; exercise 8 proves that its internal path length is 


pa i = |log,((t-1)n+1 
(n+ oz )a- “Gage? 7 [ow(¢- 1) +0). (7) 
These results have another important generalization if we shift our point 
of view slightly. Suppose that we are given m real numbers w1, wW2,..., Wm; 
the problem is to find an extended binary tree with m external nodes, and to 
associate the numbers w1,..., Wm with these nodes in such a way that the sum 


Š w;l; is minimized, where l; is the length of path from the root and the sum is 
taken over all external nodes. For example, if the given numbers are 2, 3, 4, 11, 
we can form extended binary trees such as these three: 


Here the “weighted” path lengths $- w,l; are 34, 53, and 40, respectively. (There- 
fore a perfectly balanced tree does not give the minimum weighted path length 
when the weights are 2, 3, 4, and 11, although we have seen that it does give the 
minimum in the special case wy = wz = ++: = Wm = 1.) 

Several interpretations of weighted path length arise in connection with 
different computer algorithms; for example, we can apply it to the merging of 
sorted sequences of respective lengths w1, w2,...,Wm (see Chapter 5). One of 
the most straightforward applications of this idea is to consider a binary tree 
as a general search procedure, where we start at the root and then make some 
test; the outcome of the test sends us to one of the two branches, where we may 
make further tests, etc. For example, if we want to decide which of four different 
alternatives is true, and if these possibilities will be true with the respective 
probabilities oat z, a and H, a tree that minimizes the weighted path length 
will constitute an optimal search procedure. [These are the weights shown in (8), 
times a scale factor.] 

An elegant algorithm for finding a tree with minimum weighted path length 
was discovered by D. Huffman [Proc. IRE 40 (1952), 1098-1101]: First find the 
two w’s of lowest value, say wı and w2. Then solve the problem for m—1 weights 
Ww, + W2, W3, ..., Wm, and replace the node 


wit we (9) 
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- ae ; (10) 


As an example of Huffman’s method, let us find the optimal tree for the 
weights 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41. First we combine 2 + 3, 
and look for the solution to 5,5,7,...,41; then we combine 5 + 5, etc. The 
computation is summarized as follows: 


2 3 5 7 11 13 #17 19 23 29 31 37 4 


in this solution by 


5 & 7 11 13 17 19 23 29 31 37 4l 
10 7 11 13 17 19 23 29 31 37 41 
17 11 13 17 19 23 29 31 37 41 

17 24 17 19 23 29 31 37 41 

24 34 19 23 29 31 37 41 

24 34 42 29 31 37 41 

34 42 53 31 37 41 

42 53 65 37 4l 

42 53 65 78 

95 65 78 

95 143 

238 


Therefore the following tree corresponds to Huffman’s construction: 


(The numbers inside the circular nodes show the correspondence between this 
tree and our computation; see also exercise 9.) 

It is not hard to prove that this method does in fact minimize the weighted 
path length, by induction on m. Suppose we have wı < wg < w3 < -+ < Wm, 
where m > 2, and suppose that we are given a tree that minimizes the weighted 
path length. (Such a tree certainly exists, since only finitely many binary trees 
with m terminal nodes are possible.) Let V be an internal node of maximum 
distance from the root. If wı and wz are not the weights already attached to the 
children of V, we can interchange them with the values that are already there; 
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such an interchange does not increase the weighted path length. Thus there is 
a tree that minimizes the weighted path length and contains the subtree (10). 
Now it is easy to prove that the weighted path length of a tree for the weights 
W1, .--, Wm that contains (10) as a subtree is minimized if and only if that tree 
with (10) replaced by (9) has minimum path length for the weights w1 + we, ws, 
1+, Wm. (See exercise 9.) 

Every time this construction combines two weights, they are at least as big as 
the weights previously combined, if the given w; were nonnegative. This means 
that there is a neat way to find Huffman’s tree, provided that the given weights 
have been sorted into nondecreasing order: We simply maintain two queues, one 
containing the original weights and the other containing the combined weights. 
At each step the smallest unused weight will appear at the front of one of the 
queues, so we never have to search for it. See exercise 13, which shows that the 
same idea works even when the weights might be negative. 

In general, there are many trees that minimize 57 w,l,;. If the algorithm 
sketched in the preceding paragraph always uses an original weight instead of a 
combined weight in case of ties, then the tree it constructs has the smallest value 
of max l; and of X, l; among all trees that minimize J- w,l,;. If the weights are 
positive, this tree actually minimizes J` w; f(l,;) for any convex function f, over 
all such trees. [See E. S. Schwartz, Information and Control 7 (1964), 37-44; 
G. Markowsky, Acta Informatica 16 (1981), 363-370.] 

Huffman’s method can be generalized to t-ary trees as well as binary trees. 
(See exercise 10.) Another important generalization of Huffman’s method is 
discussed in Section 6.2.2. Further discussion of path length appears in Sections 
5.3.1, 5.4.9, and 6.3. 


EXERCISES 


1. [12] Are there any other binary trees with 12 internal nodes and minimum path 
length, besides the complete binary tree (5)? 


2. [17] Draw an extended binary tree with terminal nodes containing the weights 
1, 4, 9, 16, 25, 36, 49, 64, 81, 100, having minimum weighted path length. 

3. [M24] An extended binary tree with m external nodes determines a set of path 
lengths lı, l2,...,lm that describe the lengths of paths from the root to the respective 
external nodes. Conversely, if we are given a set of numbers l1, l2,...,lm, is it always 
possible to construct an extended binary tree in which these numbers are the path 
lengths in some order? Show that this is possible if and only if ja Smir = 1, 

4. [M25] (E. S. Schwartz and B. Kallick.) Assume that wi < w2 < --- < wm. Show 
that there is an extended binary tree that minimizes X` wl; and for which the terminal 
nodes in left to right order contain the respective values w1, w2,...,Wm. [For example, 
tree (11) does not meet this condition since the weights appear in the order 19, 23, 
11, 18, 29, 2, 3, 5, 7, 17, 31, 37, 41. We seek a tree for which the weights appear in 
ascending order, and this does not always happen with Huffman’s construction.] 


5. [HM26] Let 


B(w,z) = 5 bnpw?z”, 


n,p20 
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where bnp is the number of binary trees with n nodes and internal path length p. [Thus, 


B(w,z) =1+ 2+ 2wz? + (w? + 4w?)z? + (4w* + 2w® + 8w®) 24 +---; 


B(1, z) is the function B(z) of Eq. (13) in Section 2.3.4.4.] 

a) Find a functional relation that characterizes B(w, z), generalizing 2.3.4.4-(12). 

b) Use the result of (a) to determine the average internal path length of a binary tree 
1 2n 


with n nodes, assuming that each of the mat ( n) trees is equally probable. 


c) Find the asymptotic value of this quantity. 
6. [16] If a t-ary tree is extended with square nodes as in (1), what is the relation 
between the number of square and circular nodes corresponding to Eq. (2)? 


7. [M21] What is the relation between external and internal path length in a t-ary 
tree? (See exercise 6; a generalization of Eq. (3) is desired.) 

8. [M23] Prove Eq. (7). 

9. [M21] The numbers that appear in the circular nodes of (11) are equal to the 


sums of the weights in the external nodes of the corresponding subtree. Show that the 
sum of all values in the circular nodes is equal to the weighted path length. 


10. [M26] (D. Huffman.) Show how to construct a t-ary tree with minimum weighted 
path length, given nonnegative weights w1, W2,..., Wm. Construct an optimal ternary 
tree for weights 1, 4, 9, 16, 25, 36, 49, 64, 81, 100. 

11. [16] Is there any connection between the complete binary tree (5) and the “Dewey 


decimal notation” for binary trees described in exercise 2.3.1—5? 


12. [M20] Suppose that a node has been chosen at random in a binary tree, with each 
node equally likely. Show that the average size of the subtree rooted at that node is 
related to the path length of the tree. 


13. [22] Design an algorithm that begins with m weights wi < w2 <--- < wm and 
constructs an extended binary tree having minimum weighted path length. Represent 
the final tree in three arrays 


A{i}...A[2m—1], L[j...L[m—1], Rl... Rim- 1; 


here L[i] and Rfi] point to the left and right children of internal node i, the root is 
node 1, and A/i] is the weight of node i. The original weights should appear as the 
external node weights A[m], ..., A[2m— 1]. Your algorithm should make fewer than 
2m weight-comparisons. Caution: Some or all of the given weights may be negative! 


14. [25] (T. C. Hu and A. C. Tucker.) After k steps of Huffman’s algorithm, the 
nodes combined so far form a forest of m — k extended binary trees. Prove that this 
forest has the smallest total weighted path length, among all forests of m — k extended 
binary trees that have the given weights. 

15. [M25] Show that a Huffman-like algorithm will find an extended binary tree that 
minimizes (a) max(wi + 11,...,Wm + lm); (b) wir +- + wma, given x > 1. 

16. [M25] (F. K. Hwang.) Let wi < --- < wm and wi < +- < wh, be two sets of 
weights with 


k k 
Sws v; forl<k<m. 
j=l j=l 


Prove that the minimum weighted path lengths satisfy )7"", wjlj < Uy 4 wil 


t 
J 
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17. [HM30] (C. R. Glassey and R. M. Karp.) Let sı, ..., Sm —1 be the numbers 
inside the internal (circular) nodes of an extended binary tree formed by Huffman’s 
algorithm, in the order of construction. Let si, ..., 8%,—1 be the internal node weights 
of any extended binary tree on the same set of weights {wi,...,Wm}, listed in any 
order such that each nonroot internal node appears before its parent. (a) Prove that 
Y isj < = , 8; for 1 < k < m. (b) The result of (a) is equivalent to 


m-1 m—1 


2 t(s) )< SO f(s}) 


a. 
ll 
un 


for every nondecreasing concave eae f, namely every function f with f'(x) > 0 
and f(x) < 0. [See Hardy, Littlewood, and Pólya, Messenger of Math. 58 (1929), 
145-152.] Use this fact to show that the minimum value in the recurrence 


F(n) = f(n) + min (F(R) + F(n—k)), FQ) =0 


always occurs when k = glle(n/3)1 given any function f(n) with the property that 
Af(n) = f(n +1) — f(n) > 0 and A? f(n) = Af(n +1) — Af(n) <0 


*2.3.4.6. History and bibliography. Trees have of course been in existence 
since the third day of creation, and through the ages tree structures (especially 
family trees) have been in common use. The concept of tree as a formally defined 
mathematical entity seems to have appeared first in the work of G. Kirchhoff 
[Annalen der Physik und Chemie 72 (1847), 497-508, English translation in 
IRE Transactions CT-5 (1958), 4-7]; Kirchhoff used free trees to find a set of 
fundamental cycles in an electrical network in connection with the law that bears 
his name, essentially as we did in Section 2.3.4.1. The concept also appeared at 
about the same time in the book Geometrie der Lage (pages 20-21) by K. G. Chr. 
von Staudt. The name “tree” and many results dealing mostly with enumeration 
of trees began to appear ten years later in a series of papers by Arthur Cayley [see 
Collected Mathematical Papers of A. Cayley 3 (1857), 242-246; 4 (1859), 112- 
115; 9 (1874), 202-204; 9 (1875), 427-460; 10 (1877), 598-600; 11 (1881), 365- 
367; 13 (1889), 26-28]. Cayley was unaware of the previous work of Kirchhoff 
and von Staudt; his investigations began with studies of the structure of algebraic 
formulas, and they were later inspired chiefly by applications to the problem of 
isomers in chemistry. Tree structures were also studied independently by C. W. 
Borchardt [Crelle 57 (1860), 111-121]; J. B. Listing [Göttinger Abhandlungen, 
Math. Classe, 10 (1862), 137-139]; and C. Jordan [Crelle 70 (1869), 185-190]. 

The “infinity lemma” was formulated first by Dénes König [Fundamenta 
Math. 8 (1926), 114-134], and he gave it a prominent place in his classic book 
Theorie der endlichen und unendlichen Graphen (Leipzig: 1936), Chapter 6. 
A similar result called the “fan theorem” occurred slightly earlier in the work 
of L. E. J. Brouwer [Verhandelingen Akad. Amsterdam 12 (1919), 7], but this 
involved much stronger hypotheses; see A. Heyting, Intuitionism (1956), Sec- 
tion 3.4, for a discussion of Brouwer’s work. 

Formula (3) of Section 2.3.4.4 for enumerating unlabeled oriented trees was 
given by Cayley in his first paper on trees. In his second paper he enumerated 
unlabeled ordered trees; an equivalent problem in geometry (see exercise 1) 


v 
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had already been proposed and solved by L. Euler, who mentioned his results 
in a letter to C. Goldbach on 4 September 1751 [see J. von Segner and L. 
Euler, Novi Commentarii Academiæ Scientiarum Petropolitanæ 7 (1758-1759), 
summary 13-15, 203-210]. Euler’s problem was the subject of seven papers by 
G. Lamé, E. Catalan, O. Rodrigues, and J. Binet in Journal de mathématiques 
3, 4 (1838, 1839); additional references appear in the answer to exercise 2.2.1—4. 
The corresponding numbers are now commonly called “Catalan numbers.” A 
Mongolian Chinese mathematician, An-T’u Ming, had encountered the Catalan 
numbers before 1750 in his study of infinite series, but he did not relate them to 
trees or other combinatorial objects [see J. Luo, Acta Scientiarum Naturalium 
Universitatis Intramongolicee 19 (1988), 239-245; Combinatorics and Graph 
Theory (World Scientific Publishing, 1993), 68-70]. Catalan numbers occur in 
an enormous number of different contexts; Richard Stanley explains more than 
60 of them in his magnificent book Enumerative Combinatorics 2 (Cambridge 
Univ. Press, 1999), Chapter 6. Perhaps most surprising of all is the Catalan 
connection to certain arrangements of numbers that H. S. M. Coxeter has called 
“frieze patterns” because of their symmetry; see exercise 4. 

The formula n”~? for the number of labeled free trees was discovered by J. J. 
Sylvester [Quart. J. Pure and Applied Math. 1 (1857), 55-56], as a byproduct 
of his evaluation of a certain determinant (exercise 2.3.4.2-28). Cayley gave 
an independent derivation of the formula in 1889 [see the reference above]; 
his discussion, which was extremely vague, hinted at a connection between 
labeled oriented trees and (n—1)-tuples of numbers. An explicit correspondence 
demonstrating such a connection was first published by Heinz Priifer [Arch. 
Math. und Phys. 27 (1918), 142-144], quite independently of Cayley’s prior work. 
A large literature on this subject has developed, and the classical results are 
surveyed beautifully in J. W. Moon’s book, Counting Labelled Trees (Montreal: 
Canadian Math. Congress, 1970). 

A very important paper on the enumeration of trees and many other kinds 
of combinatorial structures was published by G. Pólya in Acta Math. 68 (1937), 
145-253. For a discussion of enumeration problems for graphs and an excellent 
bibliography of the early literature, see the survey by Frank Harary in Graph 
Theory and Theoretical Physics (London: Academic Press, 1967), 1-41. 

The principle of minimizing weighted path length by repeatedly combining 
the smallest weights was discovered by D. Huffman [Proc. IRE 40 (1952), 1098- 
1101], in connection with the design of codes for minimizing message lengths. 
The same idea was independently published by Seth Zimmerman [AMM 66 
(1959), 690-693]. 

Several other noteworthy papers about the theory of tree structures have 
been cited in Sections 2.3.4.1 through 2.3.4.5 in connection with particular topics. 


EXERCISES 

1. [21] Find a simple one-to-one correspondence between binary trees with n nodes 
and dissections of an (n + 2)-sided convex polygon into n triangles, assuming that the 
sides of the polygon are distinct. 
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> 2. [M26] T.P. Kirkman conjectured in 1857 that the number of ways to draw k non- 
overlapping diagonals in an r-sided polygon is Ca) (77°) /(r+k). 
a) Extend the correspondence of exercise 1 to obtain an equivalent problem about 
the enumeration of trees. 
b) Prove Kirkman’s conjecture by using the methods of exercise 2.3.4.4—-32. 
> 3. [M30] Consider all ways of partitioning the vertices of a convex n-gon into k non- 
empty parts, in such a way that no diagonal between two vertices of one part crosses 
a diagonal between two vertices of another part. 
a) Find a one-to-one correspondence between noncrossing partitions and an interest- 
ing class of tree structures. 
b) Given n and k, how many ways are there to make such a partition? 


> 4. [M38] (Conway and Coxeter.) A frieze pattern is an infinite array such as 
1 1 1 1-1-1-1. 1-1 Pe Ia OL 


1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 


in which the top and bottom rows consist entirely of 1s, and each diamond of adjacent 
values a : d satisfies ad — bc = 1. Find a one-to-one correspondence between n-node 
binary trees and (n + 1)-rowed frieze patterns of positive integers. 


2.3.5. Lists and Garbage Collection 


Near the beginning of Section 2.3 we defined a List informally as “a finite 
sequence of zero or more atoms or Lists.” 
Any forest is a List; for example, 


vie X 
h 


may be regarded as the List 
(a: (b, c, d), e: (F, g: (h))), (2) 


and the corresponding List diagram would be 


ea ~~ 
JN, 7 o) 


| 
h 


The reader should review at this point the introduction to Lists given earlier, 
in particular (3), (4), (5), (6), (7) in the opening pages of Section 2.3. Recall that, 
in (2) above, the notation “a: (b, c, d)” means that (b, c, d) is a List of three atoms, 
which has been labeled with the attribute “a”. This convention is compatible 
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with our general policy that each node of a tree may contain information besides 
its structural connections. However, as was discussed for trees in Section 2.3.3, 
it is quite possible and sometimes desirable to insist that all Lists be unlabeled, 
so that all the information appears in the atoms. 

Although any forest may be regarded as a List, the converse is not true. 
The following List is perhaps more typical than (2) and (3) since it shows how 
the restrictions of tree structure might be violated: 


L= (a: N,b,c: (d: N),e: L), N = (f:(), 9: (h: L, j: N)) (4) 


which may be diagrammed as 


fe  g* di 
/ N 
hL] jÙ] 


[Compare with the example in 2.3-(7). The form of these diagrams need not be 
taken too seriously.] 

As we might expect, there are many ways to represent List structures within 
a computer memory. These methods are usually variations on the same basic 
theme by which we have used binary trees to represent general forests of trees: 
One field, say RLINK, is used to point to the next element of a List, and another 
field DLINK may be used to point to the first element of a sub-List. By a natural 
extension of the memory representation described in Section 2.3.2, we would 
represent the List (5) as follows: 


Ael e4 —>b (atom) ° >clele `e ° 


Unfortunately, this simple idea is not quite adequate for the most common 
List processing applications. For example, suppose that we have the List L = 
(A, a, (A, A)), which contains three references to another List A = (b,c, d). One 
of the typical List processing operations is to remove the leftmost element of A, 
so that A becomes (c,d); but this requires three changes to the representation 
of L, if we are to use the technique shown in (6), since each pointer to A points 
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to the element b that is being deleted. A moment’s reflection will convince the 
reader that it is extremely undesirable to change the pointers in every reference 
to A just because the first element of A is being deleted. (In this example we 
could try to be tricky, assuming that there are no pointers to the element c, by 
copying the entire element c into the location formerly occupied by b and then 
deleting the old element c. But this trick fails to work when A loses its last 
element and becomes empty.) 

For this reason the representation scheme (6) is generally replaced by another 
scheme that is similar, but uses a List head to begin each List, as was introduced 
in Section 2.2.4. Each List contains an additional node called its List head, so 
that the configuration (6) would, for example, be represented thus: 


a ay 
7 e Ja e| >b (atom) | -}+—>] c| ẹ | >} e ° 


e 


C : 


The introduction of such header nodes is not really a waste of memory space 
in practice, since many uses for the apparently unused fields — the shaded areas 
in diagram (7) — generally present themselves. For example, there is room for a 
reference count, or a pointer to the right end of the List, or an alphabetic name, 
or a “scratch” field that aids traversal algorithms, etc. 

In our original diagram (6), the node containing b is an atom while the node 
containing f specifies an empty List. These two things are structurally identical, 
so the reader would be quite justified in asking why we bother to talk about 
“atoms” at all; with no loss of generality we could have defined Lists as merely 
“a finite sequence of zero or more Lists,” with our usual convention that each 
node of a List may contain data besides its structural information. This point 
of view is certainly defensible and it makes the concept of an “atom” seem very 
artificial. There is, however, a good reason for singling out atoms as we have 
done, when efficient use of computer memory is taken into consideration, since 
atoms are not subject to the same sort of general-purpose manipulation that is 
desired for Lists. The memory representation (6) shows there is probably more 
room for information in an atomic node, b, than in a List node, f; and when 
List head nodes are also present as in (7), there is a dramatic difference between 
the storage requirements for the nodes b and f. Thus the concept of atoms is 
introduced primarily to aid in the effective use of computer memory. Typical 


\ 
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Lists contain many more atoms than our example would indicate; the example of 
(4)-(7) is intended to show the complexities that are possible, not the simplicities 
that are usual. 

A List is in essence nothing more than a linear list whose elements may 
contain pointers to other Lists. The common operations we wish to perform on 
Lists are the usual ones desired for linear lists (creation, destruction, insertion, 
deletion, splitting, concatenation), plus further operations that are primarily of 
interest for tree structures (copying, traversal, input and output of nested infor- 
mation). For these purposes any of the three basic techniques for representing 
linked linear lists in memory — namely straight, circular, or double linkage — can 
be used, with varying degrees of efficiency depending on the algorithms being 
employed. For these three types of representation, diagram (7) might appear in 
memory as follows: 


Memory Straight linkage Circular linkage Double linkage 
location INFO DLINK RLINK | INFO DLINK RLINK | INFO DLINK LLINK RLINK 
010: — head 020 — head 020 — head 050 020 
020: 060 030 060 030 060 010 030 
030: atom 040 atom 040 atom 020 040 
040: 090 050 090 050 090 030 050 
050: 010 A 010 010 010 040 010 


a a a 
b b b 
c c c 
e e e 
060: — head 070 — head 070 — head 080 070 (8) 
070: f 110 080 f 110 080 f 110 060 080 
080: g 120 A g 120 060 g 120 070 060 
090: — head 100 — head 100 — head 100 100 
100: d 060 A d 060 090 d 060 090 090 


110: head A head 110 head 110 110 
120: — head 130 — head 130 — head 140 130 
130: h 010 140 h 010 140 h 010 120 140 
140: j 060 A j 060 120 j 060 130 120 


Here “LLINK” is used for a pointer to the left in a doubly linked representation. 
The INFO and DLINK fields are identical in all three forms. 

There is no need to repeat here the algorithms for List manipulation in any 
of these three forms, since we have already discussed the ideas many times. The 
following important points about Lists, which distinguish them from the simpler 
special cases treated earlier, should however be noted: 

1) It is implicit in the memory representation above that atomic nodes are 
distinguishable from nonatomic nodes; furthermore, when circular or doubly 
linked Lists are being used, it is desirable to distinguish header nodes from the 
other types, as an aid in traversing the Lists. Therefore each node generally 
contains a TYPE field that tells what kind of information the node represents. 
This TYPE field is often used also to distinguish between various types of atoms 
(for example, between alphabetic, integer, or floating point quantities, for use 
when manipulating or displaying the data). 

2) The format of nodes for general List manipulation with the MIX computer 
might be designed in one of the following two ways. 
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a) Possible one-word format, assuming that all INFO appears in atoms: 


s |T| REF /RLINK (9) 


S (sign): Mark bit used in garbage collection (see below). 
T (type): T = 0 for List head; T = 1 for sub-List element; T > 1 for atoms. 


REF: When T = 0, REF is a reference count (see below); when T = 1, 
REF points to the List head of the sub-List in question; when 
T > 1, REF points to a node containing a mark bit and five bytes 
of atomic information. 


RLINK: Pointer for straight or circular linkage as in (8). 


b) Possible two-word format: 


S | T |LLINK|RLINK 


INFO 
S, T: As in (9). 
LLINK, RLINK: The usual pointers for double linkage as in (8). 
INFO: A full word of information associated with this node; for a 


header node this may include a reference count, a running 
pointer to the interior of the List to facilitate linear traversal, 
an alphabetic name, etc. When T = 1 this information 
includes DLINK. 


3) It is clear that Lists are very general structures; indeed, it seems fair to 
state that any structure whatsoever can be represented as a List when appropri- 
ate conventions are made. Because of this universality of Lists, a large number 
of programming systems have been designed to facilitate List manipulation, and 
there are usually several such systems available at any computer installation. 
Such systems are based on a general-purpose format for nodes such as (9) or 
(10) above, designed for flexibility in List operations. Actually, it is clear that 
this general-purpose format is usually not the best format suited to a partic- 
ular application, and the processing time using the general-purpose routines 
is noticeably slower than a person would achieve by hand-tailoring the system 
to a particular problem. For example, it is easy to see that nearly all of the 
applications we have worked out so far in this chapter would be encumbered by 
a general-List representation as in (g) or (10) instead of the node format that was 
given in each case. A List manipulation routine must often examine the T-field 
when it processes nodes, and that was not needed in any of our programs so 
far. This loss of efficiency is repaid in many instances by the comparative ease 
of programming and the reduction of debugging time when a general-purpose 
system is used. 

4) There is also an extremely significant difference between algorithms for 
List processing and the algorithms given previously in this chapter. Since a single 
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List may be contained in many other Lists, it is by no means clear exactly when 
a List should be returned to the pool of available storage. Our algorithms so far 
have always said “AVAIL < X”, whenever NODE(X) was no longer needed. But 
since general Lists can grow and die in a completely unpredictable manner, it is 
often quite difficult to tell just when a particular node is superfluous. Therefore 
the problem of maintaining the list of available space is considerably more 
difficult with Lists than it was in the simple cases considered previously. We will 
devote the rest of this section to a discussion of the storage reclamation problem. 

Let us imagine that we are designing a general-purpose List processing 
system that will be used by hundreds of other programmers. Two principal 
methods have been suggested for maintaining the available space list: the use 
of reference counters, and garbage collection. The reference-counter technique 
makes use of a new field in each node, which contains a count of how many 
arrows point to this node. Such a count is rather easy to maintain as a program 
runs, and whenever it drops to zero, the node in question becomes available. 
The garbage-collection technique, on the other hand, requires a new one-bit field 
in each node called the mark bit. The idea in this case is to write nearly all 
the algorithms so that they do not return any nodes to free storage, and to let 
the program run merrily along until all of the available storage is gone; then a 
“recycling” algorithm makes use of the mark bits to identify all nodes that are 
not currently accessible and to return them to available storage, after which the 
program can continue. 

Neither of these two methods is completely satisfactory. The principal 
drawback of the reference-counter method is that it does not always free all 
the nodes that are available. It works fine for overlapped Lists (Lists that 
contain common sub-Lists); but recursive Lists, like our examples L and N in 
(4), will never be returned to storage by the reference-counter technique. Their 
counts will be nonzero (since they refer to themselves) even when no other List 
accessible to the running program points to them. Furthermore, the reference- 
counter method uses a good chunk of space in each node (although this space is 
sometimes available anyway due to the computer word size). 

The difficulty with the garbage-collection technique, besides the annoying 
loss of a bit in each node, is that it runs very slowly when nearly all the memory 
space is in use; and in such cases the number of free storage cells found by the 
reclamation process is not worth the effort. Programs that exceed the capacity 
of storage (and many undebugged programs do!) often waste a good deal of time 
calling the garbage collector several almost fruitless times just before storage is 
finally exhausted. A partial solution to this problem is to let the programmer 
specify a number k, signifying that processing should not continue after a garbage 
collection run has found k or fewer free nodes. 

Another problem is the occasional difficulty of determining exactly what 
Lists are not garbage at a given stage. If the programmer has been using any 
nonstandard techniques or keeping any pointer values in unusual places, chances 
are good that the garbage collector will go awry. Some of the greatest mysteries 
in the history of debugging have been caused by the fact that garbage collection 
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suddenly took place at an unexpected time during the running of programs 
that had worked many times before. Garbage collection also requires that 
programmers keep valid information in all pointer fields at all times, although 
we often find it convenient to leave meaningless information in fields that the 
program doesn’t use— for example, the link in the rear node of a queue; see 
exercise 2.2.3-6. 

Although garbage collection requires one mark bit for each node, we could 
keep a separate table of all the mark bits packed together in another memory 
area, with a suitable correspondence between the location of a node and its mark 
bit. On some computers this idea can lead to a method of handling garbage 
collection that is more attractive than giving up a bit in each node. 

J. Weizenbaum has suggested an interesting modification of the reference- 
counter technique. Using doubly linked List structures, he puts a reference 
counter only in the header of each List. Thus, when pointer variables traverse 
a List, they are not included in the reference counts for the individual nodes. If 
we know the rules by which reference counts are maintained for entire Lists, we 
know (in theory) how to avoid referring to any List that has a reference count 
of zero. We also have complete freedom to explicitly override reference counts 
and to return particular Lists to available storage. These ideas require careful 
handling; they prove to be somewhat dangerous in the hands of inexperienced 
programmers, and they’ve tended to make program debugging more difficult due 
to the consequences of referring to nodes that have been erased. The nicest part 
of Weizenbaum’s approach is his treatment of Lists whose reference count has just 
gone to zero: Such a List is appended at the end of the current available list — 
this is easy to do with doubly linked Lists— and it is considered for available 
space only after all previously available cells are used up. Eventually, as the 
individual nodes of this List do become available, the reference counters of Lists 
they refer to are decreased by one. This delayed action of erasing Lists is quite 
efficient with respect to running time; but it tends to make incorrect programs 
run correctly for awhile! For further details see CACM 6 (1963), 524-544. 

Algorithms for garbage collection are quite interesting for several reasons. 
In the first place, such algorithms are useful in other situations when we want to 
mark all nodes that are directly or indirectly referred to by a given node. (For 
example, we might want to find all subroutines called directly or indirectly by a 
certain subroutine, as in exercise 2.2.3-26.) 

Garbage collection generally proceeds in two phases. We assume that the 
mark bits of all nodes are initially zero (or we set them all to zero). Now the first 
phase marks all the nongarbage nodes, starting from those that are immediately 
accessible to the main program. The second phase makes a sequential pass over 
the entire memory pool area, putting all unmarked nodes onto the list of free 
space. The marking phase is the most interesting, so we will concentrate our 
attention on it. Certain variations on the second phase can, however, make it 
nontrivial; see exercise 9. 

When a garbage collection algorithm is running, only a very limited amount 
of storage is available to control the marking procedure. This intriguing problem 
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will become clear in the following discussion; it is a difficulty that is not appre- 
ciated by most people when they first hear about the idea of garbage collection, 
and for several years there was no good solution to it. 

The following marking algorithm is perhaps the most obvious. 


Algorithm A (Marking). Let the entire memory used for List storage be 
NODE(1), NODE(2), ..., NODE(M), and suppose that these words either are atoms 
or contain two link fields ALINK and BLINK. Assume that all nodes are initially 
unmarked. The purpose of this algorithm is to mark all of the nodes that can be 
reached by a chain of ALINK and/or BLINK pointers in nonatomic nodes, starting 
from a set of “immediately accessible” nodes, that is, nodes pointed to by certain 
fixed locations in the main program; these fixed pointers are used as a source for 
all memory accesses. 
A1. [Initialize.] Mark all nodes that are immediately accessible. Set K + 1. 
A2. [Does NODE(K) imply another?| Set K1 + K+ 1. If NODE(K) is an atom 
or unmarked, go to step A3. Otherwise, if NODE(ALINK(K)) is unmarked: 
Mark it and, if it is not an atom, set K1 + min(K1, ALINK(K)). Similarly, 
if NODE(BLINK(K)) is unmarked: Mark it and, if it is not an atom, set 
Ki + min(K1, BLINK (K)). 
A3. [Done?] Set K «+ K1. If K <M, return to step A2; otherwise the algorithm 
terminates. J 


Throughout this algorithm and the ones that follow in this section, we will 
assume for convenience that the nonexistent node “NODE(A)” is marked. (For 
example, ALINK(K) or BLINK(K) may equal A in step A2.) 


A variant of Algorithm A sets K1 + M+ 1 in step Al, removes the operation 
“K1 4+ K + 1” from step A2, and instead changes step A3 to 


A3’. [Done?] Set K + K+1. If K <M, return to step A2. Otherwise if K1 < M, 
set K + K1 and Ki + M+1 and return to step A2. Otherwise the algorithm 
terminates. I 

It is very difficult to give a precise analysis of Algorithm A, or to determine 
whether it is better or worse than the variant just described, since no meaningful 
way to describe the probability distribution of the input presents itself. We can 
say that it takes up time proportional to nM in the worst case, where n is the 
number of cells it marks; and, in general, we can be sure that it is very slow 
when n is large. Algorithm A is too slow to make garbage collection a usable 
technique. 

Another fairly evident marking algorithm is to follow all paths and to record 
branch points on a stack as we go: 


Algorithm B (Marking). This algorithm achieves the same effect as Algo- 

rithm A, using STACK[1], STACK[2], ... as auxiliary storage to keep track of all 

paths that have not yet been pursued to completion. 

B1. [Initialize.] Let T be the number of immediately accessible nodes; mark 
them and place pointers to them in STACK[1], ..., STACK[T]. 
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B2. [Stack empty?] If T = 0, the algorithm terminates. 
B3. [Remove top entry.] Set K + STACK[T], T<- T-1. 


B4. [Examine links.] If NODE(K) is an atom, return to step B2. Otherwise, if 
NODE(ALINK(K)) is unmarked, mark it and set T +— T+ 1, STACK[T] + 
ALINK(K); if NODECBLINK(K)) is unmarked, mark it and set T + T+ 1, 
STACK[T] + BLINK(K). Return to B2. J 


Algorithm B clearly has an execution time essentially proportional to the 
number of cells it marks, and this is as good as we could possibly expect; but it 
is not really usable for garbage collection because there is no place to keep the 
stack! It does not seem unreasonable to assume that the stack in Algorithm B 
might grow up to, say, five percent of the size of memory; but when garbage 
collection is called, and all available space has been used up, there is only a 
fixed (rather small) number of cells to use for such a stack. Most of the early 
garbage collection procedures were essentially based on this algorithm. If the 
special stack space was used up, the entire program had to be terminated. 

A somewhat better alternative is possible, using a fixed stack size, by com- 
bining Algorithms A and B: 


Algorithm C (Marking). This algorithm achieves the same effect as Algo- 
rithms A and B, using an auxiliary table of H cells, STACK[0], STACK[1], ..., 
STACK [H — 1]. 

In this algorithm, the action “insert X on the stack” means the following: 
“Set T << (T+1) mod H, and STACK[T] «+ X. If T = B, set B + (B+ 1) modH and 
K1 + min(K1, STACK[B]).” (Note that T points to the current top of the stack, 
and B points one place below the current bottom; STACK essentially operates as 
an input-restricted deque.) 

C1. [Initialize.] Set T + H— 1, B + H— 1, K1 + M+1. Mark all the immediately 
accessible nodes, and successively insert their locations onto the stack (as 
just described above). 

C2. [Stack empty?] If T = B, go to C5. 

C3. [Remove top entry.] Set K + STACK[T], T + (T — 1) mod H. 


C4. [Examine links.] If NODE(K) is an atom, return to step C2. Otherwise, if 
NODE(ALINK(K)) is unmarked, mark it and insert ALINK(K) on the stack. 
Similarly, if NODE(BLINK(K)) is unmarked, mark it and insert BLINK(K) on 
the stack. Return to C2. 


C5. [Sweep.] If K1 >M, the algorithm terminates. (The variable K1 represents 
the smallest location where there is a possibility of a new lead to a node 
that should be marked.) Otherwise, if NODE(K1) is an atom or unmarked, 
increase K1 by 1 and repeat this step. If NODE(K1) is marked, set K + K1, 
increase Ki by 1, and go to C4. I 


This algorithm and Algorithm B can be improved if X is never put on the 
stack when NODE(X) is an atom; moreover, steps B4 and C4 need not put items 
on the stack when they know that the items will immediately be removed. Such 
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modifications are straightforward and they have been left out to avoid making 
the algorithms unnecessarily complicated. 

Algorithm C is essentially equivalent to Algorithm A when H = 1, and to 
Algorithm B when H = M; it gradually becomes more efficient as H becomes 
larger. Unfortunately, Algorithm C defies a precise analysis for the same reason 
as Algorithm A, and we have no good idea how large H should be to make this 
method fast enough. It is plausible but uncomfortable to say that a value like 
H = 50 is sufficient to make Algorithm C usable for garbage collection in most 
applications. 

Algorithms B and C use a stack kept in sequential memory locations; but we 
have seen earlier in this chapter that linked memory techniques are well suited to 
maintaining stacks that are not consecutive in memory. This suggests the idea 
that we might keep the stack of Algorithm B somehow scattered through the 
same memory area in which we are collecting garbage. This could be done easily 
if we were to give the garbage collection routine a little more room in which 
to breathe. Suppose, for example, we assume that all Lists are represented as 
in (9), except that the REF fields of List head nodes are used for garbage collection 
purposes instead of as reference counts. We can then redesign Algorithm B so 
that the stack is maintained in the REF fields of the header nodes: 


Algorithm D (Marking). This algorithm achieves the same effect as Algorithms 
A, B, and C, but it assumes that the nodes have S, T, REF, and RLINK fields as 
described above, instead of ALINKs and BLINKs. The S field is used as the mark 
bit, so that S(P) = 1 means that NODE (P) is marked. 


D1. [Initialize.] Set TOP + A. Then for each pointer P to the head of an 
immediately accessible List (see step A1 of Algorithm A), if S(P) = 0, set 
S(P) + 1, REF (P) + TOP, TOP + P. 


D2. [Stack empty?] If TOP = A, the algorithm terminates. 
D3. [Remove top entry.] Set P + TOP, TOP + REF (P). 


D4. [Move through List.] Set P + RLINK(P); then if P = A, or if T(P) = 0, go 
to D2. Otherwise set S(P) «+ 1. If T(P) > 1, set SCREF(P)) + 1 (thereby 
marking the atomic information). Otherwise (T(P) = 1), set Q + REF (P); 
if Q Æ A and S(Q) = 0, set S(Q) + 1, REF(Q) + TOP, TOP + Q. Repeat 
step D4. I 


Algorithm D may be compared to Algorithm B, which is quite similar, and 
its running time is essentially proportional to the number of nodes marked. 
However, Algorithm D is not recommended without qualification, because its 
seemingly rather mild restrictions are often too stringent for a general List- 
processing system. This algorithm essentially requires that all List structures 
be well-formed, as in (7), whenever garbage collection is called into action. 
But algorithms for List manipulations momentarily leave the List structures 
malformed, and a garbage collector such as Algorithm D must not be used during 
those momentary periods. Moreover, care must be taken in step D1 when the 
program contains pointers to the middle of a List. 
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Fig. 38. Algorithm E for marking with no auxiliary stack space. 


These considerations bring us to Algorithm E, which is an elegant marking 
method discovered independently by Peter Deutsch and by Herbert Schorr and 
W. M. Waite in 1965. The assumptions used in this algorithm are just a little 
different from those of Algorithms A through D. 


Algorithm E (Marking). Assume that a collection of nodes is given having the 
following fields: 


MARK (a one-bit field), 

ATOM (another one-bit field), 
ALINK (a pointer field), 
BLINK (a pointer field). 


When ATOM = 0, the ALINK and BLINK fields may contain A or a pointer to 
another node of the same format; when ATOM = 1, the contents of the ALINK and 
BLINK fields are irrelevant to this algorithm. 

Given a nonnull pointer PO, this algorithm sets the MARK field equal to 1 in 
NODE (PO) and in every other node that can be reached from NODE (PO) by a chain 
of ALINK and BLINK pointers in nodes with ATOM = MARK = 0. The algorithm 
uses three pointer variables, T, Q, and P. It modifies the links and control bits in 
such a way that all ATOM, ALINK, and BLINK fields are restored to their original 
settings after completion, although they may be changed temporarily. 

E1. [Initialize.] Set T < A, P 4+ PO. (Throughout the remainder of this 
algorithm, the variable T has a dual significance: When T Æ A, it points 
to the top of what is essentially a stack as in Algorithm D; and the node 
that T points to once contained a link equal to P in place of the “artificial” 
stack link that currently occupies NODE (T) .) 

E2. [Mark.] Set MARK (P) < 1. 

E3. [Atom?] If ATOM(P) = 1, go to E6. 

E4. [Down ALINK.] Set Q + ALINK(P). If Q # A and MARK(Q) = 0, set 
ATOM(P) + 1, ALINK(P) + T, T & P, P + Q, and go to E2. (Here the 
ATOM field and ALINK fields are temporarily being altered, so that the List 
structure in certain marked nodes has been rather drastically changed. But 
these changes will be restored in step E6.) 

E5. [Down BLINK.] Set Q + BLINK(P). If Q # A and MARK(Q) = 0, set 
BLINK(P) + T, T P, P< Q, and go to E2. 
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a ALINK(MARK] blo] . ~[1] b 
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Fig. 39. A structure marked by Algorithm E. (The table shows only changes that 
have occurred since the previous step.) 


E6. [Up.] (This step undoes the link switching made in step E4 or E5; the 
setting of ATOM(T) tells whether ALINK(T) or BLINK(T) is to be restored.) 
If T = A, the algorithm terminates. Otherwise set Q < T. If ATOM(Q) = 1, 
set ATOM(Q) «+ 0, T + ALINK(Q), ALINK(Q) + P, P + Q, and return to E5. 
If ATOM(Q) = 0, set T + BLINK (Q), BLINK (Q) + P, P + Q, and repeat E6. J 


An example of this algorithm in action appears in Fig. 39, which shows the 
successive steps encountered for a simple List structure. The reader will find it 
worthwhile to study Algorithm E very carefully; notice how the linking structure 
is artificially changed in steps E4 and E5, in order to maintain a stack analogous 
to the stack in Algorithm D. When we return to a previous state, the ATOM 
field is used to tell whether ALINK or BLINK contains the artificial address. The 
“nesting” shown at the bottom of Fig. 39 illustrates how each nonatomic node 
is visited three times during Algorithm E: The same configuration (T,P) occurs 
at the beginning of steps E2, E5, and E6. 
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A proof that Algorithm E is valid can be formulated by induction on the 
number of nodes that are to be marked. We prove at the same time that P 
returns to its initial value PO at the conclusion of the algorithm; for details, see 
exercise 3. Algorithm E will run faster if step E3 is deleted and if special tests 
for “ATOM(Q) = 1” and appropriate actions are made in steps E4 and E5, as 
well as a test “ATOM(PO) = 1” in step El. We have stated the algorithm in its 
present form for simplicity; the modifications just stated appear in the answer 
to exercise 4. 

The idea used in Algorithm E can be applied to problems other than garbage 
collection; in fact, its use for tree traversal has already been mentioned in 
exercise 2.3.1-21. The reader may also find it useful to compare Algorithm E 
with the simpler problem solved in exercise 2.2.3-—7. 

Of all the marking algorithms we have discussed, only Algorithm D is 
directly applicable to Lists represented as in (9). The other algorithms all 
test whether or not a given node P is an atom, and the conventions of (9) are 
incompatible with such tests because they allow atomic information to fill an 
entire word except for the mark bit. However, each of the other algorithms 
can be modified so that they will work when atomic data is distinguished from 
pointer data in the word that links to it instead of by looking at the word 
itself. In Algorithms A or C we can simply avoid marking atomic words until 
all nonatomic words have been properly marked; then one further pass over all 
the data suffices to mark all the atomic words. Algorithm B is even easier to 
modify, since we need merely keep atomic words off the stack. The adaptation of 
Algorithm E is almost as simple, although if both ALINK and BLINK are allowed 
to point to atomic data it will be necessary to introduce another 1-bit field in 
nonatomic nodes. This is generally not hard to do. (For example, when there 
are two words per node, the least significant bit of each link field may be used 
to store temporary information.) 

Although Algorithm E requires a time proportional to the number of nodes 
it marks, this constant of proportionality is not as small as in Algorithm B; 
the fastest garbage collection method known combines Algorithms B and E, as 
discussed in exercise 5. 

Let us now try to make some quantitative estimates of the efficiency of 
garbage collection, as opposed to the philosophy of “AVAIL < X” that was 
used in most of the previous examples in this chapter. In each of the previous 
cases we could have omitted all specific mention of returning nodes to free 
space and we could have substituted a garbage collector instead. (In a special- 
purpose application, as opposed to a set of general-purpose List manipulation 
subroutines, the programming and debugging of a garbage collector is more 
difficult than the methods we have used, and, of course, garbage collection 
requires an extra bit reserved in each node; but we are interested here in the 
relative speed of the programs once they have been written and debugged.) 

The best garbage collection routines known have an execution time essen- 
tially of the form c,N + coM, where cı and c2 are constants, N is the number of 
nodes marked, and M is the total number of nodes in the memory. Thus M — N is 
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the number of free nodes found, and the amount of time required to return these 
nodes to free storage is (cıN + coM)/(M—N) per node. Let N = pM; this figure 
becomes (c1p+c2)/(1—p). So if p = 3, that is, if the memory is three-fourths full, 
we spend 3c; + 4c2 units of time per free node returned to storage; when p = i, 
the corresponding cost is only ic + $09. If we do not use the garbage collection 
technique, the amount of time per node returned is essentially a constant, cs, 
and it is doubtful that c3/c, will be very large. Hence we can see to what extent 
garbage collection is inefficient when the memory becomes full, and how it is 
correspondingly efficient when the demand on memory is light. 

Many programs have the property that the ratio p = N/M of good nodes 
to total memory is quite small. When the pool of memory becomes full in such 
cases, it might be best to move all the active List data to another memory pool of 
equal size, using a copying technique (see exercise 10) but without bothering to 
preserve the contents of the nodes being copied. Then when the second memory 
pool fills up, we can move the data back to the first one again. With this method 
more data can be kept in high-speed memory at once, because link fields tend 
to point to nearby nodes. Moreover, there’s no need for a marking phase, and 
storage allocation is simply sequential. 

It is possible to combine garbage collection with some of the other methods 
of returning cells to free storage; these ideas are not mutually exclusive, and some 
systems employ both the reference counter and the garbage collection schemes, 
besides allowing the programmer to erase nodes explicitly. The idea is to employ 
garbage collection only as a “last resort” whenever all other methods of returning 
cells have failed. An elaborate system, which implements this idea and also 
includes a mechanism for postponing operations on reference counts in order to 
achieve further efficiency, has been described by L. P. Deutsch and D. G. Bobrow 
in CACM 19 (1976), 522-526. 

A sequential representation of Lists, which saves many of the link fields at 
the expense of more complicated storage management, is also possible. See N. E. 
Wiseman and J. O. Hiles, Comp. J. 10 (1968), 338-343; W. J. Hansen, CACM 
12 (1969), 499-507; and C. J. Cheney, CACM 13 (1970), 677-678. 

Daniel P. Friedman and David S. Wise have observed that the reference 
counter method can be employed satisfactorily in many cases even when Lists 
point to themselves, if certain link fields are not included in the counts [Inf. Proc. 
Letters 8 (1979), 41—45]. 

A great many variants and refinements of garbage collection algorithms 
have been proposed. Jacques Cohen, in Computing Surveys 13 (1981), 341- 
367, presents a detailed review of the literature prior to 1981, with important 
comments about the extra cost of memory accesses when pages of data are 
shuttled between slow memory and fast memory. 

Garbage collection as we have described it is unsuitable for “real time” 
applications, where each basic List operation must be quick; even if the garbage 
collector goes into action infrequently, it requires large chunks of computer time 
on those occasions. Exercise 12 discusses some approaches by which real-time 
garbage collection is possible. 
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It is a very sad thing nowadays 
that there is so little useless information. 


— OSCAR WILDE (1894) 


EXERCISES 


1. [M21] In Section 2.3.4 we saw that trees are special cases of the “classical” 
mathematical concept of a directed graph. Can Lists be described in graph-theoretic 
terminology? 


2. [20] In Section 2.3.1 we saw that tree traversal can be facilitated using a threaded 
representation inside the computer. Can List structures be threaded in an analogous 
way? 

3. [M26] Prove the validity of Algorithm E. [Hint: See the proof of Algorithm 
201 | 


4. [28] Write a MIX program for Algorithm E, assuming that nodes are represented as 
one MIX word, with MARK the (0:0) field [“+” = 0, “—” = 1], ATOM the (1:1) field, ALINK 
the (2:3) field, BLINK the (4:5) field, and A = 0. Also determine the execution time 
of your program in terms of relevant parameters. (In the MIX computer the problem 
of determining whether a memory location contains —0 or +0 is not quite trivial, and 
this can be a factor in your program.) 


5. [25] (Schorr and Waite.) Give a marking algorithm that combines Algorithms B 
and E as follows: The assumptions of Algorithm E with regard to fields within the 
nodes, etc., are retained; but an auxiliary stack STACK[1], STACK[2], ..., STACK[N] is 
used as in Algorithm B, and the mechanism of Algorithm E is employed only when the 
stack is full. 


6. [00] The quantitative discussion at the end of this section says that the cost of 
garbage collection is approximately ciN + c2M units of time; where does the “coM” term 
come from? 


7. [24] (R. W. Floyd.) Design a marking algorithm that is similar to Algorithm E in 
using no auxiliary stack, except that (i) it has a more difficult task to do, because each 
node contains only MARK, ALINK, and BLINK fields—there is no ATOM field to provide 
additional control; yet (ii) it has a simpler task to do, because it marks only a binary 
tree instead of a general List. Here ALINK and BLINK are the usual LLINK and RLINK in 
a binary tree. 


8. [27] (L. P. Deutsch.) Design a marking algorithm similar to Algorithms D and E 
in that it uses no auxiliary memory for a stack, but modify the method so that it 
works with nodes of variable size and with a variable number of pointers having the 
following format: The first word of a node has two fields MARK and SIZE; the MARK field 
is to be treated as in Algorithm E, and the SIZE field contains a number n > 0. This 
means that there are n consecutive words after the first word, each containing two 
fields MARK (which is zero and should remain so) and LINK (which is A or points to the 
first word of another node). For example, a node with three pointers would comprise 
four consecutive words: 


First word MARK = 0 (will be set to 1) SIZE = 3 
Second word MARK = 0 LINK = first pointer 
Third word MARK = 0 LINK = second pointer 


Fourth word MARK = 0 LINK = third pointer. 


Your algorithm should mark all nodes reachable from a given node PO. 
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> 9. [28] (D. Edwards.) Design an algorithm for the second phase of garbage collection 
that “compacts storage” in the following sense: Let NODE(1), ..., NODE(M) be one-word 
nodes with fields MARK, ATOM, ALINK, and BLINK, as described in Algorithm E. Assume 
that MARK = 1 in all nodes that are not garbage. The desired algorithm should relocate 
the marked nodes, if necessary, so that they all appear in consecutive locations NODE(1), 
..., NODE(K), and at the same time the ALINK and BLINK fields of nonatomic nodes 
should be altered if necessary so that the List structure is preserved. 


> 10. [28] Design an algorithm that copies a List structure, assuming that an internal 
representation like that in (7) is being used. (Thus, if your procedure is asked to copy 
the List whose head is the node at the upper left corner of (7), a new set of Lists having 
14 nodes, and with structure and information identical to that shown in (7), should 
be created.) 

Assume that the List structure is stored in memory using S, T, REF, and RLINK 
fields as in (9), and that NODE (PO) is the head of the List to be copied. Assume further 
that the REF field in each List head node is A; to avoid the need for additional memory 
space, your copying procedure should make use of the REF fields (and reset them to A 
again afterwards). 


11. [M30] Any List structure can be “fully expanded” into a tree structure by repeat- 
ing all overlapping elements until none are left; when the List is recursive, this gives an 
infinite tree. For example, the List (5) would expand into an infinite tree whose first 
four levels are 


KK AT AS 


hx jx fx gx f» gx dx ax b cœ ex 
M\/\ AA INFN | AA 


Design an algorithm to test the equivalence of two List structures, in the sense 
that they have the same diagram when fully expanded. For example, Lists A and B 
are equivalent in this sense, if 


A = (a: C, b,a: (b: D) 
B = (a: (b: D), b,a: E 
C = (b: (a: C)) 
D = (a: (b: D)) 
E = (b: (a: ©) ). 


12. [30] (M. Minsky.) Show that it is possible to use a garbage collection method 
reliably in a “real time” application, for example when a computer is controlling 
some physical device, even when stringent upper bounds are placed on the maximum 
execution time required for each List operation performed. [Hint: Garbage collection 
can be arranged to work in parallel with the List operations, if appropriate care is 
taken.] 
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2.4. MULTILINKED STRUCTURES 


Now THAT WE have examined linear lists and tree structures in detail, the 
principles of representing structural information within a computer should be 
evident. In this section we will look at another application of these techniques, 
this time for the typical case in which the structural information is slightly more 
complicated: In higher-level applications, several types of structure are usually 
present simultaneously. 

A “multilinked structure” involves nodes with several link fields in each 
node, not just one or two as in most of our previous examples. We have already 
seen some examples of multiple linkage, such as the simulated elevator system 
in Section 2.2.5 and the multivariate polynomials in Section 2.3.3. 

We shall see that the presence of many different kinds of links per node does 
not necessarily make the accompanying algorithms any more difficult to write 
or to understand than the algorithms already studied. We will also discuss the 
important question, “How much structural information ought to be explicitly 
recorded in memory?” 

The problem we will consider arises in connection with writing a compiler 
program for the translation of COBOL and related languages. A programmer who 
uses COBOL may give alphabetic names to program variables on several levels; 
for example, the program might refer to files of data for sales and purchases, 
having the following structure: 


1 SALES 1 PURCHASES 
2 DATE 2 DATE 
3 MONTH 3 DAY 
3 DAY 3 MONTH 
3 YEAR 3 YEAR 
2 TRANSACTION 2 TRANSACTION 
3 ITEM 3 ITEM (1) 
3 QUANTITY 3 QUANTITY 
3 PRICE 3 PRICE 
3 TAX 3 TAX 
3 BUYER 3 SHIPPER 
4 NAME 4 NAME 
4 ADDRESS 4 ADDRESS 


This configuration indicates that each item in SALES consists of two parts, the 
DATE and the TRANSACTION; the DATE is further divided into three parts, and the 
TRANSACTION likewise has five subdivisions. Similar remarks apply to PURCHASES. 
The relative order of these names indicates the order in which the quantities 
appear in external representations of the file (for example, magnetic tape or 
printed forms); notice that in this example “DAY” and “MONTH” appear in opposite 
order in the two files. The programmer also gives further information, not shown 
in this illustration, that tells how much space each item of information occupies 
and in what format it appears; such considerations are not relevant to us in this 
section, so they will not be mentioned further. 
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A COBOL programmer first describes the file layout and the other program 
variables, then specifies the algorithms that manipulate those quantities. To 
refer to an individual variable in the example above, it would not be sufficient 
merely to give the name DAY, since there is no way of telling if the variable called 
DAY is in the SALES file or in the PURCHASES file. Therefore a COBOL programmer 
is given the ability to write “DAY OF SALES” to refer to the DAY part of a SALES 
item. The programmer could also write, more completely, 


“DAY OF DATE OF SALES”, 


but in general there is no need to give more qualification than necessary to avoid 
ambiguity. Thus, 


“NAME OF SHIPPER OF TRANSACTION OF PURCHASES” 


may be abbreviated to 
“NAME OF SHIPPER” 


since only one part of the data has been called SHIPPER. 
These rules of COBOL may be stated more precisely as follows: 


a) Each name is immediately preceded by an associated positive integer called 
its level number. A name either refers to an elementary item or it is the 
name of a group of one or more items whose names follow. In the latter 
case, each item of the group must have the same level number, which must 
be greater than the level number of the group name. (For example, DATE 
and TRANSACTION above have level number 2, which is greater than the level 
number 1 of SALES.) 

b) To refer to an elementary item or group of items named Ap, the general 
form is 


Ao OF A; OF ... OF An, 


where n > 0 and where, for 0 < j < n, Aj is the name of some item 
contained directly or indirectly within a group named Aj; ;. There must be 
exactly one item Ag satisfying this condition. 

c) If the same name Ao appears in several places, there must be a way to refer 
to each use of the name by using qualification. 


As an example of rule (c), the data configuration 


1 AA 
2 BB 
3 cc (2) 
3 DD 
2 cC 


would not be allowed, since there is no unambiguous way to refer to the second 
appearance of CC. (See exercise 4.) 
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COBOL has another feature that affects compiler writing and the application 
we are considering, namely an option in the language that makes it possible to 
refer to many items at once. A COBOL programmer may write 


MOVE CORRESPONDING a TO 8 


which moves all items with corresponding names from data area a to data area 8. 
For example, the COBOL statement 


MOVE CORRESPONDING DATE OF SALES TO DATE OF PURCHASES 


would mean that the values of MONTH, DAY, and YEAR from the SALES file are to 
be moved to the variables MONTH, DAY, and YEAR in the PURCHASES file. (The 
relative order of DAY and MONTH is thereby interchanged.) 

The problem we will investigate in this section is to design three algorithms 
suitable for use in a COBOL compiler, which are to do the following things: 


Operation 1. To process a description of names and level numbers such 
as (1), putting the relevant information into tables within the compiler for use 
in operations 2 and 3. 

Operation 2. To determine if a given qualified reference, as in rule (b), is 
valid, and when it is valid to locate the corresponding data item. 

Operation 3. To find all corresponding pairs of items indicated by a given 
CORRESPONDING statement. 


We will assume that our compiler already has a “symbol table subroutine” 
that will convert an alphabetic name into a link that points to a table entry for 
that name. (Methods for constructing symbol table algorithms are discussed in 
detail in Chapter 6.) In addition to the Symbol Table, there is a larger table 
that contains one entry for each item of data in the COBOL source program that 
is being compiled; we will call this the Data Table. 

Clearly, we cannot design an algorithm for operation 1 until we know what 
kind of information is to be stored in the Data Table, and the form of the Data 
Table depends on what information we need in order to perform operations 2 
and 3; thus we look first at operations 2 and 3. 

In order to determine the meaning of the COBOL reference 


Ao OF A, OF... OF Anp, n> 0, (3) 


we should first look up the name Ap in the Symbol Table. There ought to be 
a series of links from the Symbol Table entry to all Data Table entries for this 
name. Then for each Data Table entry we will want a link to the entry for the 
group item that contains it. Now if there is a further link field from the Data 
Table items back to the Symbol Table, it is not hard to see how a reference like 
(3) can be processed. Furthermore, we will want some sort of links from the 
Data Table entries for group items to the items in the group, in order to locate 
the pairs indicated by “MOVE CORRESPONDING”. 
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We have thereby found a potential need for five link fields in each Data 
Table entry: 


PREV (a link to the previous entry with the same name, if any); 
PARENT (a link to the smallest group, if any, containing this item); 
NAME (a link to the Symbol Table entry for this item); 

CHILD (a link to the first subitem of a group); 

SIB (a link to the next subitem in the group containing this item). 


It is clear that COBOL data structures like those for SALES and PURCHASES above 
are essentially trees; and the PARENT, CHILD, and SIB links that appear here are 
familiar from our previous study. (The conventional binary tree representation 
of a tree consists of the CHILD and SIB links; adding the PARENT link gives what 
we have called a “triply linked tree.” The five links above consist of these three 
tree links together with PREV and NAME, which superimpose further information 
on the tree structure.) 


Perhaps not all five of these links will turn out to be necessary, or sufficient, 
but we will try first to design our algorithms under the tentative assumption 
that Data Table entries will involve these five link fields (plus further information 
irrelevant to our problems). As an example of the multiple linking used, consider 
the two COBOL data structures 


E 
D 
9G 


aN N 
o1 

O o aAa WO A 
A, 
A 
x 


They would be represented as shown in (5) (with links indicated symbolically). 
The LINK field of each Symbol Table entry points to the most recently encoun- 
tered Data Table entry for the symbolic name in question. 

The first algorithm we require is one that builds the Data Table in such 
a form. Note the flexibility in choice of level numbers that is allowed by the 
COBOL rules; the left structure in (4) is completely equivalent to 


1A 
2 B 
3C 
3D 
2 E 
2 F 
3G 


because level numbers do not have to be sequential. 
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Symbol Table Data Table 

LINK PREV PARENT NAME CHILD SIB 

A: Al Al: A A A B3 H1 

B: B5 B3: A Al B C7 E3 

C: C5 CT: A B3 C A D7 

D: D9 D7: A B3 D A A 

E: E9 E3: A Al E A F3 

F: F5 F3: A Al F G4 A 
G: C9 G4: A F3 G A A (5) 

H H1 H1: A A H F5 A 

o F5: | F3 H1 F G8 B5 

A information EEEE 

not relevant here Bs: | B3 H1 B A C5 

C5: | C7 H1 C E9 A 

E9: | E3 C5 E A D9 

D9: | D7 C5 D A G9 

G9: | G8 C5 G A A 


Some sequences of level numbers are illegal, however; for example, if the 
level number of D in (4) were changed to “6” (in either place) we would have 
a meaningless data configuration, violating the rule that all items of a group 
must have the same number. The following algorithm therefore makes sure that 
COBOL/’s rule (a) has not been broken. 


Algorithm A (Build Data Table). This algorithm is given a sequence of pairs 
(L,P), where L is a positive integer “level number” and P points to a Symbol 
Table entry, corresponding to COBOL data structures such as (4) above. The 
algorithm builds a Data Table as in the example (5) above. When P points to a 
Symbol Table entry that has not appeared before, LINK(P) will equal A. This 
algorithm uses an auxiliary stack that is treated as usual (using either sequential 
memory allocation, as in Section 2.2.2, or linked allocation, as in Section 2.2.3). 


A1. [Initialize.] Set the stack contents to the single entry (0,A). (The stack 
entries throughout this algorithm are pairs (L,P), where L is an integer 
and P is a pointer; as this algorithm proceeds, the stack contains the level 
numbers and pointers to the most recent data entries on all levels higher in 
the tree than the current level. For example, just before encountering the 
pair “3 F” in the example above, the stack would contain 


(0,4) (1,A1) (3, E8) 
from bottom to top.) 
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A2. [Next item.] Let (L,P) be the next data item from the input. If the input 
is exhausted, however, the algorithm terminates. Set Q < AVAIL (that is, 
let Q be the location of a new node in which we can put the next Data Table 
entry). 


A3. [Set name links.] Set 
PREV(Q) + LINK(P), LINK(P) + Q, NAME(Q) +P. 


(This properly sets two of the five links in NODE(Q). We now want to set 
PARENT, CHILD, and SIB appropriately.) 


A4. [Compare levels.] Let the top entry of the stack be (L1,P1). If L1 < L, set 
CHILD(P1) + Q (or, if P1 = A, set FIRST + Q, where FIRST is a variable 
that will point to the first Data Table entry) and go to A6. 


A5. [Remove top level.] If L1 > L, remove the top stack entry, let (L1,P1) be the 
new entry that has just come to the top of the stack, and repeat step A5. If 
L1 < L, signal an error (mixed numbers have occurred on the same level). 
Otherwise, namely when L1 = L, set SIB(P1) + Q, remove the top stack 
entry, and let (L1,P1) be the pair that has just come to the top of the stack. 


A6. [Set family links.] Set PARENT(Q) + P1, CHILD(Q) + A, SIB(Q) + A. 
AT. [Add to stack.] Place (L,Q) on top of the stack, and return to step A2. I 


The introduction of an auxiliary stack, as explained in step Al, makes this 
algorithm so transparent that it needs no further explanation. 
The next problem is to locate the Data Table entry corresponding to a 
reference 
Ao OF A; OF ... OF An, n> 0. (6) 


A good compiler will also check to ensure that such a reference is unambiguous. 
In this case, a suitable algorithm suggests itself immediately: All we need to do 
is to run through the list of Data Table entries for the name Ag and make sure 
that exactly one of these entries matches the stated qualification A,,..., An- 


Algorithm B (Check a qualified reference). Corresponding to reference (6), a 
Symbol Table subroutine will find pointers P, P,,...,P, to the Symbol Table 
entries for Ag, A1,..., An, respectively. 

The purpose of this algorithm is to examine Po, P,,...,P, and either to 
determine that reference (6) is in error, or to set variable Q to the address of the 
Data Table entry for the item referred to by (6). 


B1. [Initialize.] Set Q< A, P + LINK (Pp). 


B2. [Done?] If P = A, the algorithm terminates; at this point Q will equal A if 
(6) does not correspond to any Data Table entry. But if P 4 A, set S <P 
and k + 0. (S is a pointer variable that will run from P up the tree through 
PARENT links; k is an integer variable that goes from 0 to n. In practice, the 
pointers Po,...,P, would often be kept in a linked list, and instead of k, we 
would substitute a pointer variable that traverses this list; see exercise 5.) 
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B3. [Match complete?] If k < n go on to B4. Otherwise we have found a 
matching Data Table entry; if Q # A, this is the second entry found, so an 
error condition is signaled. Set Q + P, P + PREV(P), and go to B2. 


BA. [Increase k.] Set k + k +1. 


B5. [Move up tree.] Set S < PARENT(S). If S = A, we have failed to find a 
match; set P + PREV(P) and go to B2. 


B6. [A; match?] If NAME(S) = Pk, go to B3, otherwise go to B5. I 
Note that the CHILD and SIB links are not needed by this algorithm. 


4 J 


At top 


B4. Increase k > B5. Move up tree 


No 
B1. Initialize > Be. Done? a 


Error 


No 


Yes B6. Ay match? 


Fig. 40. Algorithm for checking a COBOL reference. 


The third and final algorithm that we need concerns “MOVE CORRESPONDING”; 
before we design such an algorithm, we must have a precise definition of what is 
required. The COBOL statement 


MOVE CORRESPONDING a TO 8 (7) 


where a and £ are references such as (6) to data items, is an abbreviation for 
the set of all statements 
MOVE a’ TO 8’ 


where there exists an integer n > 0 and n names Ao, A1,..., An—1 such that 


a’ = Ao OF A; OF... OF Ap- OF a 
B' = Ap OF A, OF ... OF Anı OF 8 


(8) 


and either a’ or 8’ is an elementary item (not a group item). Furthermore we 

require that the first levels of (8) show complete qualifications, namely that Aj;+1 

be the parent of A; for 0 < j < n — 1 and that a and £ are parents of An-—1; 

a’ and 8’ must be exactly n levels farther down in the tree than a and 8 are. 
With respect to our example (4), 


MOVE CORRESPONDING A TO H 


is therefore an abbreviation for the statements 
MOVE B OF A TO B OF H 
MOVE G OF F OF A TO G OF F OF H 
The algorithm to recognize all corresponding pairs a’, 8’ is quite interesting 
although not difficult; we move through the tree whose root is a, in preorder, 
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simultaneously looking in the 6 tree for matching names, and skipping over 
subtrees in which no corresponding elements can possibly occur. The names 
Ao,..-,An—1 of (8) are discovered in the opposite order Ay_1,..., Ao. 


Algorithm C (Find CORRESPONDING pairs). Given PO and QO, which point to 
Data Table entries for a and 8, respectively, this algorithm successively finds 
all pairs (P,Q) of pointers to items (a’, 6’) satisfying the constraints mentioned 
above. 


C1. [Initialize.] Set P + PO, Q + QO. (In the remainder of this algorithm, the 
pointer variables P and Q will walk through trees having the respective roots 
a and £.) 


C2. [Elementary?] If CHILD(P) = A or CHILD(Q) = A, output (P,Q) as one of the 
desired pairs and go to C5. Otherwise set P + CHILD(P), Q «+ CHILD(Q). 
(In this step, P and Q point to items a’ and p’ satisfying (8), and we wish to 
MOVE a’ TO p’ if and only if either a’ or 6’ (or both) is an elementary item.) 


C3. [Match name.] (Now P and Q point to data items that have respective 
complete qualifications of the forms 


Ao OF A, OF ... OF Anı OF a 


and 
Bo OF A; OF ... OF Anı OF £. 


The object is to see if we can make By = Ap by examining all the names 
of the group A; OF ... OF An—ı OF 8.) If NAME(P) = NAME(Q), go to C2 
(a match has been found). Otherwise, if SIB(Q) # A, set Q + SIB(Q) and 
repeat step C3. (If SIB(Q) = A, no matching name is present in the group, 
and we continue on to step C4.) 


C4. [Move on.] If SIB(P) # A, set P + SIB(P) and Q «+ CHILD(PARENT(Q)), 
and go back to C3. If SIB(P) = A, set P + PARENT(P) and Q «+ PARENT(Q). 


C5. [Done?] If P = PO, the algorithm terminates; otherwise go to C4. J 
A flow chart for this algorithm is shown in Fig. 41. A proof that this algorithm 


is valid can readily be constructed by induction on the size of the trees involved 
(see exercise 9). 


Q+SIB(Q) 
d Yes Y 


C2. 
Elementary? 


C1. 
Initialize 


C4. 
C3. No P<—PARENT (P) 
Match name match More 


Match P4—SIB(P) No 


Fig. 41. Algorithm for “MOVE CORRESPONDING”. 
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At this point it is worthwhile to study the ways in which the five link fields 
PREV, PARENT, NAME, CHILD, and SIB are used by Algorithms B and C. The 
striking feature is that these five links constitute a “complete set” in the sense 
that Algorithms B and C do virtually the minimum amount of work as they 
move through the Data Table. Whenever they need to refer to another Data 
Table entry, its address is immediately available; there is no need to conduct a 
search. It would be difficult to imagine how Algorithms B and C could possibly 
be made any faster if any additional link information were present in the table. 
(See exercise 11, however.) 

Each link field may be viewed as a clue to the program, planted there in 
order to make the algorithms run faster. (Of course, the algorithm that builds 
the tables, Algorithm A, runs correspondingly slower, since it has more links to 
fill in. But table-building is done only once.) It is clear, on the other hand, that 
the Data Table constructed above contains much redundant information. Let us 
consider what would happen if we were to delete certain of the link fields. 

The PREV link, while not used in Algorithm C, is extremely important for 
Algorithm B, and it seems to be an essential part of any COBOL compiler unless 
lengthy searches are to be carried out. A field that links together all items of 
the same name therefore seems essential for efficiency. We could perhaps modify 
the strategy slightly and adopt circular linking instead of terminating each list 
with A, but there is no reason to do this unless other link fields are changed or 
eliminated. 

The PARENT link is used in both Algorithms B and C, although its use in 
Algorithm C could be avoided if we used an auxiliary stack in that algorithm, or 
if we augmented SIB so that thread links are included (as in Section 2.3.2). So we 
see that the PARENT link has been used in an essential way only in Algorithm B. If 
the SIB link were threaded, so that the items that now have SIB = A would have 
SIB = PARENT instead, it would be possible to locate the parent of any data item 
by following the SIB links; the added thread links could be distinguished either 
by having a new TAG field in each node that says whether the SIB link is a thread, 
or by the condition “SIB(P) < P” if the Data Table entries are kept consecutively 
in memory in order of appearance. This would mean a short search would be 
necessary in step B5, and the algorithm would be correspondingly slower. 

The NAME link is used by the algorithms only in steps B6 and C3. In both 
cases we could make the tests “NAME(S) = Pk” and “NAME(P) = NAME(Q)” in 
other ways if the NAME link were not present (see exercise 10), but this would 
significantly slow down the inner loops of both Algorithms B and C. Here again 
we see a trade-off between the space for a link and the speed of the algorithms. 
(The speed of Algorithm C is not especially significant in COBOL compilers, when 
typical uses of MOVE CORRESPONDING are considered; but Algorithm B should be 
fast.) Experience indicates that other important uses are found for the NAME link 
within a COBOL compiler, especially in printing diagnostic information. 

Algorithm A builds the Data Table step by step, and it never has occasion 
to return a node to the pool of available storage; so we usually find that Data 
Table entries take consecutive memory locations in the order of appearance of 
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the data items in the COBOL source program. Thus in our example (5), locations 
Al, B3, ... would follow each other. This sequential nature of the Data Table 
leads to certain simplifications; for example, the CHILD link of each node is 
either A or it points to the node immediately following, so CHILD can be reduced 
to a 1-bit field. Alternatively, CHILD could be removed in favor of a test if 
PARENT(P + c) = P, where c is the node size in the Data Table. 

Thus the five link fields are not all essential, although they are helpful from 
the standpoint of speed in Algorithms B and C. This situation is fairly typical 
of most multilinked structures. 

It is interesting to note that at least half a dozen people writing COBOL 
compilers in the early 1960s arrived independently at this same way to maintain 
a Data Table using five links (or four of the five, usually with the CHILD link 
missing). The first publication of such a technique was by H. W. Lawson, Jr. 
[ACM National Conference Digest (Syracuse, N.Y.: 1962)]. But in 1965 an 
ingenious technique for achieving the effects of Algorithms B and C, using only 
two link fields and sequential storage of the Data Table, without a very great 
decrease in speed, was introduced by David Dahm; see exercises 12 through 14. 


EXERCISES 


1. [00] Considering COBOL data configurations as tree structures, are the data items 
listed by a COBOL programmer in preorder, postorder, or neither of those orders? 


2. [10] Comment about the running time of Algorithm A. 


3. [22] The PL/I language accepts data structures like those in COBOL, except that 
any sequence of level numbers is possible. For example, the sequence 


1A 1A 
3 B 2 B 
5c is equivalent to 3c 
4D 3D 
2E 2E 


In general, rule (a) is modified to read, “The items of a group must have a sequence 
of nonincreasing level numbers, all of which are greater than the level number of the 
group name.” What modifications to Algorithm A would change it from the COBOL 
convention to this PL/I convention? 


4. [26] Algorithm A does not detect the error if a COBOL programmer violates rule 
(c) stated in the text. How should Algorithm A be modified so that only data structures 
satisfying rule (c) will be accepted? 


5. [20] In practice, Algorithm B may be given a linked list of Symbol Table references 


as input, instead of what we called “Po, Pi,..., Pn.” Let T be a pointer variable such 
that 
INFO(T) = Po, INFO(RLINK(T)) =P, ..., INFO(RLINK!!(T)) = Pa, RLINK” +! (T) = A. 


Show how to modify Algorithm B so that it uses such a linked list as input. 


6. [23] The PL/I language accepts data structures much like those in COBOL, but 
does not make the restriction of rule (c); instead, we have the rule that a qualified 
reference (3) is unambiguous if it shows “complete” qualification— that is, if Aj41 is 
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the parent of A; for 0 < j < n, and if A, has no parent. Rule (c) is now weakened 
to the simple condition that no two items of a group may have the same name. The 
second “CC” in (2) would be referred to as “CC OF AA” without ambiguity; the three 
data items 1A 


2A 
3 A 
would be referred to as “A”, “A OF A”, “A OF A OF A” with respect to the PL/I con- 
vention just stated. [Note: Actually the word “OF” is replaced by a period in PL/I, 
and the order is reversed; “CC OF AA” is really written “AA.CC” in PL/I, but this is not 
important for the purposes of the present exercise.] Show how to modify Algorithm B 
so that it follows the PL/I convention. 


7. [15] Given the data structures in (1), what does the COBOL statement “MOVE 
CORRESPONDING SALES TO PURCHASES” mean? 


8. [10] Under what circumstances is “MOVE CORRESPONDING a TO 8” exactly the 
same as “MOVE a TO 8”, according to the definition in the text? 


9. [M23] Prove that Algorithm C is correct. 


10. [23] (a) How could the test “NAME(S) = P,” in step B6 be performed if there were 
no NAME link in the Data Table nodes? (b) How could the test “NAME(P) = NAME(Q)” 
in step C3 be performed if there were no NAME link in the Data Table entries? (Assume 
that all other links are present as in the text.) 


11. [23] What additional links or changes in the strategy of the algorithms of the text 
could make Algorithm B or Algorithm C faster? 


12. [25] (D. M. Dahm.) Consider representing the Data Table in sequential locations 
with just two links for each item: 


PREV (as in the text); 
SCOPE (a link to the last elementary item in this group). 


We have SCOPE(P) = P if and only if NODE(P) represents an elementary item. For 
example, the Data Table of (5) would be replaced by 


PREV SCOPE PREV SCOPE PREV SCOPE 
Al: A G4 F3: A G4 B5: B3 B5 
B3: A D7 G4: A G4 C5: C7 G9 
C7: A C7 H1: A G9 E9: E3 E9 
D7: A D7 F5: F3 G8 D9: D7 D9 
E3: A E3 G8: G4 G8 G9: G8 G9 


(Compare with (5) of Section 2.3.3.) Notice that NODE(P) is part of the tree below 
NODE (Q) if and only if Q < P < SCOPE(Q). Design an algorithm that performs the 
function of Algorithm B when the Data Table has this format. 


13. [24] Give an algorithm to substitute for Algorithm A when the Data Table is to 
have the format shown in exercise 12. 


14. [28] Give an algorithm to substitute for Algorithm C when the Data Table has 
the format shown in exercise 12. 
15. [25] (David S. Wise.) Reformulate Algorithm A so that no extra storage is used 


for the stack. [Hint: The SIB fields of all nodes pointed to by the stack are A in the 
present formulation. ] 
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2.5. DYNAMIC STORAGE ALLOCATION 


WE HAVE SEEN how the use of links implies that data structures need not be 
sequentially located in memory; a number of tables may independently grow and 
shrink in a common pooled memory area. However, our discussions have always 
tacitly assumed that all nodes have the same size—that every node occupies a 
certain fixed number of memory cells. 

For a great many applications, a suitable compromise can be found so that 
a uniform node size is indeed used for all structures (for example, see exercise 2). 
Instead of simply taking the maximum size that is needed and wasting space in 
smaller nodes, it is customary to pick a rather small node size and to employ 
what may be called the classical linked-memory philosophy: “If there isn’t room 
for the information here, let’s put it somewhere else and plant a link to it.” 

For a great many other applications, however, a single node size is not 
reasonable; we often wish to have nodes of varying sizes sharing a common 
memory area. Putting this another way, we want algorithms for reserving and 
freeing variable-size blocks of memory from a larger storage area, where these 
blocks are to consist of consecutive memory locations. Such techniques are 
generally called dynamic storage allocation algorithms. 

Sometimes, often in simulation programs, we want dynamic storage alloca- 
tion for nodes of rather small sizes (say one to ten words); and at other times, 
often in operating systems, we are dealing primarily with rather large blocks of 
information. These two points of view lead to slightly different approaches to 
dynamic storage allocation, although the methods have much in common. For 
uniformity in terminology between these two approaches, we will generally use 
the terms block and area rather than “node” in this section, to denote a set of 
contiguous memory locations. 

In 1975 or so, several authors began to call the pool of available memory 
a “heap.” But in the present series of books, we will use that word only in its 
more traditional sense related to priority queues (see Section 5.2.3). 


A. Reservation. Figure 42 shows a typical memory map or “checkerboard,” a 
chart showing the current state of some memory pool. In this case the memory is 
shown partitioned into 53 blocks of storage that are “reserved,” or in use, mixed 
together with 21 “free” or “available” blocks that are not in use. After dynamic 
storage allocation has been in operation for awhile, the computer memory will 
perhaps look something like this. Our first problem is to answer two questions: 


a) How is this partitioning of available space to be represented inside the 
computer? 

b) Given such a representation of the available spaces, what is a good algorithm 
for finding a block of n consecutive free spaces and reserving them? 


The answer to question (a) is, of course, to keep a list of the available space 
somewhere; this is almost always done best by using the available space itself to 
contain such a list. (An exception is the case when we are allocating storage for 
a disk file or other memory in which nonuniform access time makes it better to 
maintain a separate directory of available space.) 
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Fig. 42. A memory map. 


Thus, we can link together the available segments: The first word of each 
free storage area can contain the size of that block and the address of the next 
free area. The free blocks can be linked together in increasing or decreasing order 
of size, or in order of memory address, or in essentially random order. 

For example, consider Fig. 42, which illustrates a memory of 131,072 words, 
addressed from 0 to 131071. If we were to link together the available blocks in 
order of memory location, we would have one variable AVAIL pointing to the 
first free block (in this case AVAIL would equal 0), and the other blocks would 
be represented as follows: 


location SIZE LINK 
0 101 632 


632 42 1488 


: [17 similar entries] 
73654 1909 77519 
77519 53553 A [special marker for last link] 


Thus locations 0 through 100 form the first available block; after the reserved 
areas 101-290 and 291-631 shown in Fig. 42, we have more free space in location 
632-673; etc. 

As for question (b), if we want n consecutive words, clearly we must locate 
some block of m > n available words and reduce its size to m— n. (Furthermore, 
when m = n, we must also delete this block from the list.) There may be several 
blocks with n or more cells, and so the question becomes, which area should be 
chosen? 

Two principal answers to this question suggest themselves: We can use the 
best-fit method or the first-fit method. In the former case, we decide to choose 
an area with m cells, where m is the smallest value present that is n or more. 
This might require searching the entire list of available space before a decision 
can be made. The first-fit method, on the other hand, simply chooses the first 
area encountered that has > n words. 

Historically, the best-fit method was widely used for several years; this 
naturally appears to be a good policy since it saves the larger available areas 
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for a later time when they might be needed. But several objections to the best- 
fit technique can be raised: It is rather slow, since it involves a fairly long search; 
if best-fit is not substantially better than first-fit for other reasons, this extra 
searching time is not worthwhile. More importantly, the best-fit method tends 
to increase the number of very small blocks, and proliferation of small blocks is 
usually undesirable. There are certain situations in which the first-fit technique 
is demonstrably better than the best-fit method; for example, suppose we are 
given just two available areas of memory, of sizes 1300 and 1200, and suppose 
there are subsequent requests for blocks of sizes 1000, 1100, and 250: 


memory available areas, available areas, 
request first-fit best-fit 
— 1300, 1200 1300, 1200 (1) 
1000 300, 1200 1300, 200 
1100 300, 100 200, 200 
250 50, 100 stuck 


(A contrary example appears in exercise 7.) The point is that neither method 
clearly dominates the other, hence the simple first-fit method can be recom- 
mended. 


Algorithm A (First-fit method). Let AVAIL point to the first available block 

of storage, and suppose that each available block with address P has two fields: 

SIZE(P), the number of words in the block; and LINK(P), a pointer to the next 

available block. The last pointer is A. This algorithm searches for and reserves 

a block of N words, or reports failure. 

A1. [Initialize.] Set Q < LOC(AVAIL). (Throughout the algorithm we use two 

pointers, Q and P, which are generally related by the condition P = LINK(Q). 

We assume that LINK(LOC(AVAIL)) = AVAIL.) 

A2. [End of list?] Set P + LINK(Q). If P = A, the algorithm terminates 

unsuccessfully; there is no room for a block of N consecutive words. 

A3. [Is SIZE enough?] If SIZE(P) > N, go to A4; otherwise set Q + P and return 

to step A2. 

A4. [Reserve N.] Set K + SIZE(P) —N. If K = 0, set LINK(Q) + LINK(P) 
(thereby removing an empty area from the list); otherwise set SIZE(P) + K. 
The algorithm terminates successfully, having reserved an area of length N 
beginning with locationP+K. J 


This algorithm is certainly straightforward enough. However, a significant 
improvement in its running speed can be made with only a rather slight change 
in strategy. This improvement is quite important, and the reader will find it a 
pleasure to discover it without being told the secret (see exercise 6). 

Algorithm A may be used whether storage allocation is desired for small N 
or large N. Let us assume temporarily, however, that we are primarily interested 
in large values of N. Then notice what happens when SIZE(P) is equal to N+1 in 
that algorithm: We get to step A4 and reduce SIZE(P) to 1. In other words, an 
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available block of size 1 has just been created; this block is so small it is virtually 
useless, and it just clogs up the system. We would have been better off if we had 
reserved the whole block of N + 1 words, instead of saving the extra word; it is 
often better to expend a few words of memory to avoid handling unimportant 
details. Similar remarks apply to blocks of N + K words when K is very small. 

If we allow the possibility of reserving slightly more than N words it will be 
necessary to remember how many words have been reserved, so that later when 
this block becomes available again the entire set of N + K words is freed. This 
added amount of bookkeeping means that we are tying up space in every block 
in order to make the system more efficient only in certain circumstances when 
a tight fit is found; so the strategy doesn’t seem especially attractive. However, 
a special control word as the first word of each variable-size block often turns 
out to be desirable for other reasons, and so it is usually not unreasonable to 
expect the SIZE field to be present in the first word of all blocks, whether they 
are available or reserved. 

In accordance with these conventions, we would modify step A4 above to 
read as follows: 


A4’. [Reserve > N.] Set K + SIZE(P) — N. If K < c (where c is a small positive 
constant chosen to reflect an amount of storage we are willing to sacrifice in 
the interests of saving time), set LINK(Q) + LINK(P) and L + P. Otherwise 
set SIZE(P) «+ K, L + P +K, SIZE(L) + N. The algorithm terminates 
successfully, having reserved an area of length N or more beginning with 
location L. 


A value for the constant c of about 8 or 10 is suggested, although very little 
theory or empirical evidence exists to compare this with other choices. When 
the best-fit method is being used, the test of K < c is even more important than 
it is for the first-fit method, because tighter fits (smaller values of K) are much 
more likely to occur, and the number of available blocks should be kept as small 
as possible for that algorithm. 


B. Liberation. Now let’s consider the inverse problem: How should we return 
blocks to the available space list when they are no longer needed? 

It is perhaps tempting to dismiss this problem by using garbage collection 
(see Section 2.3.5); we could follow a policy of simply doing nothing until space 
runs out, then searching for all the areas currently in use and fashioning a new 
AVAIL list. 

The idea of garbage collection is not to be recommended, however, for all 
applications. In the first place, we need a fairly “disciplined” use of pointers if we 
are to be able to guarantee that all areas currently in use will be easy to locate, 
and this amount of discipline is often lacking in the applications considered here. 
Secondly, as we have seen before, garbage collection tends to be slow when the 
memory is nearly full. 

There is another more important reason why garbage collection is not satis- 
factory, due to a phenomenon that did not confront us in our previous discussion 
of the technique: Suppose that there are two adjacent areas of memory, both 
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of which are available, but because of the garbage-collection philosophy one of 
them (shown shaded) is not in the AVAIL list. 
mm á o 


In this diagram, the heavily shaded areas at the extreme left and right are 
unavailable. We may now reserve a section of the area known to be available: 


(3) 


If garbage collection occurs at this point, we have two separate free areas, 


E å m å o 


Boundaries between available and reserved areas have a tendency to perpetuate 
themselves, and as time goes on the situation gets progressively worse. But if 
we had used a philosophy of returning blocks to the AVAIL list as soon as they 
become free, and collapsing adjacent available areas together, we would have 
collapsed (2) into 


and we would have obtained 


— 
aD 
ty 


which is much better than (4). This phenomenon causes the garbage-collection 
technique to leave memory more broken up than it should be. 

In order to remove this difficulty, we can use garbage collection together 
with the process of compacting memory, that is, moving all the reserved blocks 
into consecutive locations, so that all available blocks come together whenever 
garbage collection is done. The allocation algorithm now becomes completely 
trivial by contrast with Algorithm A, since there is only one available block at all 
times. Even though this technique takes time to recopy all the locations that are 
in use, and to change the value of the link fields therein, it can be applied with 
reasonable efficiency when there is a disciplined use of pointers, and when there 
is a spare link field in each block for use by the garbage collection algorithms. 
(See exercise 33.) 

Since many applications do not meet the requirements for the feasibility of 
garbage collection, we shall now study methods for returning blocks of memory 
to the available space list. The only difficulty in these methods is the collapsing 
problem: Two adjacent free areas should be merged into one. In fact, when an 
area bounded by two available blocks becomes free, all three areas should be 
merged together into one. In this way a good balance is obtained in memory 
even though storage areas are continually reserved and freed over a long period 
of time. (For a proof of this fact, see the “fifty-percent rule” below.) 

The problem is to determine whether the areas at either side of the returned 
block are currently available; and if they are, we want to update the AVAIL list 
properly. The latter operation is a little more difficult than it sounds. 
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The first solution to these problems is to maintain the AVAIL list in order of 
increasing memory locations. 


Algorithm B (Liberation with sorted list). Under the assumptions of Algo- 
rithm A, with the additional assumption that the AVAIL list is sorted by memory 
location (that is, if P points to an available block and LINK(P) # A, then 
LINK(P) > P), this algorithm adds the block of N consecutive cells beginning 
at location PO to the AVAIL list. We naturally assume that none of these N cells 
is already available. 


B1. [Initialize.] Set Q + LOC(AVAIL). (See the remarks in step Al above.) 


B2. [Advance P.] Set P + LINK(Q). If P = A, or if P > PO, go to B3; otherwise 
set Q + P and repeat step B2. 


B3. [Check upper bound.] If PO +N = P and P ¥ A, set N + N+SIZE(P) and 
set LINK(PO) + LINK(P). Otherwise set LINK(PO) + P. 


B4. [Check lower bound.] If Q + SIZE(Q) = PO (we assume that 


SIZE(LOC(AVAIL)) = 0, 


so this test always fails when Q = LOC(AVAIL)), set SIZE(Q) + SIZE(Q) +N 
and LINK(Q) «+ LINK(PO). Otherwise set LINK(Q) + PO, SIZE(PO)<+-N. J 


Steps B3 and B4 do the desired collapsing, based on the fact that the pointers 
Q < PO < Pare the beginning locations of three consecutive available areas. 

If the AVAIL list is not maintained in order of locations, the reader can see 
that a “brute force” approach to the collapsing problem would require a complete 
search through the entire AVAIL list; Algorithm B reduces this to a search through 
about half of the AVAIL list (in step B2) on the average. Exercise 11 shows how 
Algorithm B can be modified so that, on the average, only about one-third of the 
AVAIL list must be searched. But obviously, when the AVAIL list is long, all of 
these methods are much slower than we want them to be. Isn’t there some way 
to reserve and free storage areas so that we don’t need to do extensive searching 
through the AVAIL list? 

We will now consider a method that eliminates all searching when storage 
is returned and that can be modified, as in exercise 6, to avoid almost all of 
the searching when storage is reserved. The technique makes use of a TAG field 
at both ends of each block, and a SIZE field in the first word of each block; 
this overhead is negligible when reasonably large blocks are being used, although 
it is perhaps too much of a penalty to pay in situations when the blocks have a 
very small average size. Another method described in exercise 19 requires only 
one bit in the first word of each block, at the expense of a little more running 
time and a slightly more complicated program. 

At any rate, let us now assume that we don’t mind adding a little bit of 
control information, in order to save a good deal of time over Algorithm B when 
the AVAIL list is long. The method we will describe assumes that each block has 
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the following form: 
Reserved block (TAG = “+”) Free block (TAG = “—”) 
+| sizz] First word - | SIZE LINK 
a Second word l | LINK 


SIZE-2 words 


Pe Last word - | SIZE 


The idea in the following algorithm is to maintain a doubly linked AVAIL list, 
so that entries may conveniently be deleted from random parts of the list. The 
TAG field at either end of a block can be used to control the collapsing process, 
since we can tell easily whether or not both adjacent blocks are available. 

Double linking is achieved in a familiar way, by letting the LINK in the first 
word point to the next free block in the list, and letting the LINK in the second 
word point back to the previous block; thus, if P is the address of an available 
block, we always have 


LINK(LINK(P) +1) = P = LINK (LINK (P + 1)). (8) 


To ensure proper “boundary conditions,” the list head is set up as follows: 


LOC (AVAIL): = olo e—+> to first block in available space list 


LOC(AVAIL)+1: | - “0 | 0 e—s to last block in available space list (9) 


A first-fit reservation algorithm for this technique may be designed very 
much like Algorithm A, so we shall not consider it here (see exercise 12). The 
principal new feature of this method is the way the block can be freed in 
essentially a fixed amount of time: 


Algorithm C (Liberation with boundary tags). Assume that blocks of locations 
have the forms shown in (7), and assume that the AVAIL list is doubly linked, as 
described above. This algorithm puts the block of locations starting with address 
PO into the AVAIL list. If the pool of available storage runs from locations mo 
through mı, inclusive, the algorithm assumes for convenience that 


TAG(mp — 1) = TAG(m, +1) = “+”. 


C1. [Check lower bound.] If TAG(PO — 1) = “+”, go to C3. 


C2. [Delete lower area.] Set P + PO—SIZE(PO — 1), and then set P1 + LINK(P), 
P2 + LINK(P +1), LINK(P1 +1) <+ P2, LINK(P2) + P1, SIZE(P) + 
SIZE(P) + SIZE(PO), PO + P. 


C3. [Check upper bound.] Set P + PO + SIZE(PO). If TAG(P) = “+”, go to C5. 


C4. [Delete upper area.] Set P1 4+ LINK(P), P2 + LINK(P+1), LINK(P1+1) + 
P2, LINK(P2) + P1, SIZE(PO) + SIZE(PO) + SIZE(P), P+ P+ SIZE(P). 
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C5. [Add to AVAIL list.] Set SIZE(P — 1) 4+ SIZE(PO), LINK(PO) + AVAIL, 
LINK(PO +1) © LOC(AVAIL), LINK(AVAIL+1) < PO, AVAIL + PO, 
TAG(PO) + TAG(P—1)¢+ “—”. I 


The steps of Algorithm C are straightforward consequences of the storage 
layout (7); a slightly longer algorithm that is a little faster appears in exercise 15. 
In step C5, AVAIL is an abbreviation for LINK(LOC(AVAIL) ), as shown in (9). 


C. The “buddy system.” We will now study another approach to dynamic 
storage allocation, suitable for use with binary computers. This method uses one 
bit of overhead in each block, and it requires all blocks to be of length 1, 2, 4, 
8, or 16, etc. If a block is not 2* words long for some integer k, the next higher 
power of 2 is chosen and extra unused space is allocated accordingly. 

The idea of this method is to keep separate lists of available blocks of each 
size 2", 0 < k < m. The entire pool of memory space under allocation consists 
of 2” words, which can be assumed to have the addresses 0 through 2” — 1. 
Originally, the entire block of 2” words is available. Later, when a block of 
2* words is desired, and if nothing of this size is available, a larger available 
block is split into two equal parts; ultimately, a block of the right size 2% will 
appear. When one block splits into two (each of which is half as large as the 
original), these two blocks are called buddies. Later when both buddies are 
available again, they coalesce back into a single block; thus the process can be 
maintained indefinitely, unless we run out of space at some point. 

The key fact underlying the practical usefulness of this method is that if we 
know the address of a block (the memory location of its first word), and if we 
also know the size of that block, we know the address of its buddy. For example, 
the buddy of the block of size 16 beginning in binary location 101110010110000 
is a block starting in binary location 101110010100000. To see why this must be 
true, we first observe that as the algorithm proceeds, the address of a block of 
size 2* is a multiple of 2". In other words, the address in binary notation has at 
least k zeros at the right. This observation is easily justified by induction: If it 
is true for all blocks of size 2**1, it is certainly true when such a block is halved. 

Therefore a block of size, say, 32 has an address of the form zz... x00000 
(where the x’s represent either 0 or 1); if it is split, the newly formed buddy blocks 
have the addresses xx... x00000 and wa... 210000. In general, let buddy,(a) = 
address of the buddy of the block of size 2* whose address is x; we find that 


ge+2*, if gmod2*t1=0; 
eee l r—2¥, if emod 2+! = 2, (io 


This function is readily computed with the “exclusive or” instruction (sometimes 
called “selective complement” or “add without carry”) usually found on binary 
computers; see exercise 28. 

The buddy system makes use of a one-bit TAG field in each block: 


TAG(P) = 0, if the block with address P is reserved; w 
TAG(P) = 1, if the block with address P is available. 
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This TAG field is present in all blocks, and it must not be tampered with by 
the users who reserve blocks. The available blocks also have two link fields, 
LINKF and LINKB, which are the usual forward and backward links of a doubly 
linked list; and they also have a KVAL field to specify k when their size is 2". 
The algorithms below make use of the table locations AVAIL[0], AVAIL[1], ..., 
AVAIL[m], which serve respectively as the heads of the lists of available storage 
of sizes 1, 2, 4, ..., 2”. These lists are doubly linked, so as usual the list heads 
contain two pointers (see Section 2.2.5): 


AVAILF[k] = LINKF (LOC(AVAIL[A])) = link to rear of AVAIL[K] list; es 
AVAILB[k] = LINKB(LOC(AVAIL[A])) = link to front of AVAIL[A] list. 


Initially, before any storage has been allocated, we have 


AVAILF [m] = AVAILB[m] = 0, 
LINKF (0) = LINKB(0) = LOC(AVAIL[m]), (13) 
TAG(O) =1,  KVAL(O) =m 


(indicating a single available block of length 2™, beginning in location 0), and 
AVAILF [k] = AVAILB[k] = LOC(AVAIL[K]), forO<k<m (14) 


(indicating empty lists for available blocks of lengths 2” for all k < m). 

From this description of the buddy system, the reader may find it enjoyable 
to design the necessary algorithms for reserving and freeing storage areas before 
looking at the algorithms given below. Notice the comparative ease with which 
blocks can be halved in the reservation algorithm. 


Algorithm R (Buddy system reservation). This algorithm finds and reserves 
a block of 2” locations, or reports failure, using the organization of the buddy 
system as explained above. 


R1. [Find block.] Let j be the smallest integer in the range k < j < m for 
which AVAILF[j] Æ LOC(AVAIL[j]), that is, for which the list of available 
blocks of size 2/ is not empty. If no such j exists, the algorithm terminates 
unsuccessfully, since there are no known available blocks of sufficient size to 
meet the request. 


R2. [Remove from list.] Set L «+ AVAILB[j], P + LINKB(L), AVAILB[j] + P, 
LINKF(P) < LOC(AVAIL[j]), and TAG(L) + 0. 


R3. [Split required?] If j = k, the algorithm terminates (we have found and 
reserved an available block starting at address L). 


RA. [Split.] Decrease j by 1. Then set P + L+2/, TAG(P) + 1, KVAL(P) & j, 
LINKF(P) + LINKB(P) + LOC(AVAIL[j]), AVAILF[j] + AVAILB[j] + P. 
(This splits a large block and enters the unused half in the AVAIL[,] list, 
which was empty.) Go back to step R3. J 


Algorithm S (Buddy system liberation). This algorithm returns a block of 2” 
locations, starting in address L, to free storage, using the organization of the 
buddy system as explained above. 
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S1. [Is buddy available?] Set P + buddy; (L). (See Eq. (10).) If k = m or if 
TAG(P) = 0, or if TAG(P) = 1 and KVAL(P) Æ k, go to S3. 


S2. [Combine with buddy.] Set 
LINKF (LINKB(P)) + LINKF (P), LINKB(LINKF(P)) < LINKB (P). 


(This removes block P from the AVAIL[k] list.) Then set k + k + 1, and if 
P <L set L & P. Return to S1. 


S3. [Put on list.] Set TAG(L) + 1, P + AVAILF [k], LINKF(L) + P, LINKB(P) <L, 
KVAL(L) + k, LINKB(L) + LOC(AVAIL[k]), AVAILF[k] +L. (This puts 
block L on the AVAIL[K] list.) 4q 


D. Comparison of the methods. The mathematical analysis of these dynamic 
storage-allocation algorithms has proved to be quite difficult, but there is one 
interesting phenomenon that is fairly easy to analyze, namely the “fifty-percent 
rule”: 
If Algorithms A and B are used continually in such a way that the system 
tends to an equilibrium condition, where there are N reserved blocks in the 
system, on the average, each equally likely to be the next one freed, and 
where the quantity K in Algorithm A takes on nonzero values (or, more 
generally, values > c as in step A4') with probability p, then the average 
number of available blocks tends to approximately ipN. 


This rule tells us approximately how long the AVAIL list will be. When the 
quantity p is near 1 — this will happen if c is very small and if the block sizes are 
infrequently equal to each other — we have about half as many available blocks 
as unavailable ones; hence the name “fifty-percent rule.” 
It is not hard to derive this rule. Consider the following memory map: 
EE EE L E 


This shows the reserved blocks divided into three categories: 


A: when freed, the number of available blocks will decrease by one; 
B: when freed, the number of available blocks will not change; 
C: when freed, the number of available blocks will increase by one. 


Now let N be the number of reserved blocks, and let M be the number of 
available ones; let A, B, and C be the number of blocks of the types identified 
above. We have 
N=A+B+C 
M=12A+Bte) (15) 
where e = 0, 1, or 2 depending on conditions at the lower and upper boundaries. 
Let us assume that N is essentially constant, but that A, B, C, and € are 
random quantities that reach a stationary distribution after a block is freed and a 
(slightly different) stationary distribution after a block is allocated. The average 
change in M when a block is freed is the average value of (C — A) /N; the average 
change in M when a block is allocated is —1 +p. So the equilibrium assumption 
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tells us that the average value of C — A — N + pN is zero. But then the average 
value of 2M is pN plus the average value of e, since 2M = N+ A—C-+e by (15). 
The fifty-percent rule follows. 

Our assumption that each deletion applies to a random reserved block will 
be valid if the lifetime of a block is an exponentially distributed random variable. 
On the other hand, if all blocks have roughly the same lifetime, this assumption 
is false; John E. Shore has pointed out that type A blocks tend to be “older” 
than type C blocks when allocations and liberations tend to have a somewhat 
first-in-first-out character, since a sequence of adjacent reserved blocks tends to 
be in order from youngest to oldest and since the most recently allocated block 
is almost never type A. This tends to produce a smaller number of available 
blocks, giving even better performance than the fifty-percent rule would predict. 
[See CACM 20 (1977), 812-820.] 

For more detailed information about the fifty-percent rule, see D. J. M. 
Davies, BIT 20 (1980), 279-288; C. M. Reeves, Comp. J. 26 (1983), 25-35; 
G. Ch. Pflug, Comp. J. 27 (1984), 328-333. 


Besides this interesting rule, our knowledge of the performance of dynamic 
storage allocation algorithms is based almost entirely on Monte Carlo exper- 
iments. Readers will find it instructive to conduct their own simulation ex- 
periments when they are choosing between storage allocation algorithms for a 
particular machine and a particular application or class of applications. The 
author carried out several such experiments just before writing this section (and, 
indeed, the fifty-percent rule was noticed during those experiments before a 
proof for it was found); let us briefly examine the methods and results of those 
experiments here. 

The basic simulation program ran as follows, with TIME initially zero and 
with the memory area initially all available: 


P1. [Tick.] Advance TIME by 1. 
P2. [Sync.] Free all blocks in the system that are scheduled to be freed at the 
current value of TIME. 


P3. [Get data.] Calculate two quantities S (a random size) and T (a random 
lifetime), based on some probability distributions, using the methods of 
Chapter 3. 


P4. [Use data.] Reserve a new block of length S, which is due to be freed at 
(TIME + T). Return to Pl. I 


Whenever TIME was a multiple of 200, detailed statistics about the performance 
of the reservation and liberation algorithms were printed. The same sequence 
of values of S and T was used for each pair of algorithms tested. After TIME 
advanced past 2000, the system usually had reached a more or less steady state 
that gave every indication of being maintained indefinitely thereafter. However, 
depending on the total amount of storage available and on the distributions of 
S and T in step P3, the allocation algorithms would occasionally fail to find 
enough space and the simulation experiment was then terminated. 
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Let C be the total number of memory locations available, and let S and T 
denote the average values of S and T in step P3. It is easy to see that the expected 
number of unavailable words of memory at any given time is ST, once TIME 
is sufficiently large. When ST was greater than about 2C in the experiments, 
memory overflow usually occurred, often before C words of memory were actually 
needed. The memory was able to become over 90 percent filled when the block 
size was small compared to C, but when the block sizes were allowed to exceed 
iC (as well as taking on much smaller values) the program tended to regard the 
memory as “full” when fewer than TO locations were actually in use. Empirical 
evidence suggests strongly that block sizes larger than O should not be used 
with dynamic storage allocation if effective operation is expected. 

The reason for this behavior can be understood in terms of the fifty-percent 
rule: If the system reaches an equilibrium condition in which the size f of an 
average free block is less than the size r of an average block in use, we can expect 
to get an unfillable request unless a large free block is available for emergencies. 
Hence f > r in a saturated system that doesn’t overflow, and we have C = 
fM +rN >rM+rN ~x (p/2+1)rN. The total memory in use is therefore 
rN < C/(p/2+ 1); when p = 1 we are unable to use more than about 2/3 of the 
memory cells. 

The experiments were conducted with three size distributions for S: 


(S1) an integer chosen uniformly between 100 and 2000; 


(52) sizes (1, 2, 4, 8, 16, 32) chosen with respective probabilities (5, 4, $, 
16? 33° 33); 

($3) sizes (10, 12, 14, 16, 18, 20, 30, 40, 50, 60, 70, 80, 90, 100, 150, 200, 
250, 500, 1000, 2000, 3000, 4000) selected with equal probability. 


The time distribution T was usually a random integer chosen uniformly between 
1 and t, for fixed t = 10, 100, or 1000. 

Experiments were also made in which T was chosen uniformly between 1 and 
min(|?U | ; 12500) in step P3, where U is the number of time units remaining 
until the next scheduled freeing of some currently reserved block in the system. 
This time distribution was meant to simulate an “almost-last-in-first-out” be- 
havior: For if T were always chosen < U, the storage allocation system would 
degenerate into simply a stack operation requiring no complex algorithms. (See 
exercise 1.) The stated distribution causes T to be chosen greater than U about 
20 percent of the time, so we have almost, but not quite, a stack operation. When 
this distribution was used, algorithms such as A, B, and C behaved much better 
than usual; there were rarely, if ever, more than two items in the entire AVAIL 
list, while there were about 14 reserved blocks. On the other hand, the buddy 
system algorithms, R and S, were slower when this distribution was used, because 
they tend to split and coalesce blocks more frequently in a stack-like operation. 
The theoretical properties of this time distribution appear to be quite difficult 
to deduce (see exercise 32). 

Figure 42, which appeared near the beginning of this section, was the 
configuration of memory at TIME = 5000, with size distribution (S71) and with 
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Fig. 43. Memory map obtained with the best-fit method. (Compare this with Fig. 42, 
which shows the first-fit method, and Fig. 44, which shows the buddy system, for the 
same sequence of storage requests.) 


the times distributed uniformly in {1,...,100}, using the first-fit method just as 
in Algorithms A and B above. For this experiment, the probability p that enters 
into the “fifty-percent rule” was essentially 1, so we would expect about half as 
many available blocks as reserved blocks. Actually Fig. 42 shows 21 available 
and 53 reserved. This does not disprove the fifty-percent rule: For example, 
at TIME = 4600 there were 25 available and 49 reserved. The configuration in 
Fig. 42 merely shows how the fifty-percent rule is subject to statistical variations. 
The number of available blocks generally ranged between 20 and 30, while the 
number of reserved blocks was generally between 45 and 55. 

Figure 43 shows the configuration of memory obtained with the same data 
as Fig. 42 but with the best-fit method used instead of the first-fit method. The 
constant c in step A4’ was set to 16, to eliminate small blocks, and as a result 
the probability p dropped to about 0.7 and there were fewer available areas. 

When the time distribution was changed to vary from 1 to 1000 instead of 
1 to 100, situations precisely analogous to those shown in Figs. 42 and 43 were 
obtained, with all appropriate quantities approximately multiplied by 10. For 
example, there were 515 reserved blocks; and 240 free blocks in the equivalent 
of Fig. 42, 176 free blocks in the equivalent of Fig. 43. 

In all experiments comparing the best-fit and first-fit methods, the latter 
always appeared to be superior. When memory size was exhausted, the first-fit 
method actually stayed in action longer than the best-fit method before memory 
overflow occurred, in most instances. 

The buddy system was also applied to the same data that led to Figs. 42 
and 43, and Fig. 44 was the result. Here, all sizes in the range 257 to 512 were 
treated as 512, those between 513 and 1024 were raised to 1024, etc. On the 
average this means that more than four thirds as much memory was requested 
(see exercise 21); the buddy system, of course, works better on size distributions 
like that of ($2) above, instead of ($1). Notice that there are available blocks 
of sizes 2°, 210, 211, 212 913 and 214 in Fig. 44. 

Simulation of the buddy system showed that it performs much better than 
might be expected. It is clear that the buddy system will sometimes allow two 
adjacent areas of the same size to be available without merging them into one 
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Fig. 44. Memory map obtained with the buddy system. (The tree structure indicates 
the division of certain large blocks into buddies of half the size. Squares indicate 
available blocks.) 


(if they are not buddies); but this situation is not present in Fig. 44 and, in 
fact, it is rare in practice. In cases where memory overflow occurred, memory 
was 95 percent reserved, and this reflects a surprisingly good allocation balance. 
Furthermore, it was very seldom necessary to split blocks in Algorithm R, or to 
merge them in Algorithm S; the tree remained much like Fig. 44, with available 
blocks on the most commonly used levels. Some mathematical results that give 
insight into this behavior, at the lowest level of the tree, have been obtained by 
P. W. Purdom, Jr., and S. M. Stigler, JACM 17 (1970), 683-697. 

Another surprise was the excellent behavior of Algorithm A after the modifi- 
cation described in exercise 6; only 2.8 inspections of available block sizes were 
necessary on the average (using size distribution ($1) and times chosen uniformly 
between 1 and 1000), and more than half of the time only the minimum value, 
one iteration, was necessary. This was true in spite of the fact that about 
250 available blocks were present. The same experiment with Algorithm A 
unmodified showed that about 125 iterations were necessary on the average 
(so about half of the AVAIL list was being examined each time); 200 or more 
iterations were found to be necessary about 20 percent of the time. 

This behavior of Algorithm A unmodified can, in fact, be predicted as a 
consequence of the fifty-percent rule. At equilibrium, the portion of memory 
containing the last half of the reserved blocks will also contain the last half of 
the free blocks; that portion will be involved half of the time when a block is 
freed, and so it must be involved in half of the allocations in order to maintain 
equilibrium. The same argument holds when one-half is replaced by any other 
fraction. (These observations are due to J. M. Robson.) 
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The exercises below include MIX programs for the two principal methods 
that are recommended as a consequence of the remarks above: (i) the boundary 
tag system, as modified in exercises 12 and 16; and (ii) the buddy system. Here 
are the approximate results: 


Time for reservation Time for liberation 


Boundary tag system: 33+7A 18, 29, 31, or 34 
Buddy system: 19+ 25R 27 + 26S 


Here A > 1 is the number of iterations necessary when searching for an available 
block that is large enough; R > 0 is the number of times a block is split in two 
(the initial difference of j — k in Algorithm R); and S > 0 is the number of times 
buddy blocks are reunited during Algorithm S. The simulation experiments 
indicate that under the stated assumptions with size distribution (S1) and time 
chosen between 1 and 1000, we may take A = 2.8, R = S = 0.04 on the average. 
(The average values A = 1.3, R = S = 0.9 were observed when the “almost-last- 
in-first-out” time distribution was substituted as explained above.) This shows 
that both methods are quite fast, with the buddy system slightly faster in MIX’s 
case. Remember that the buddy system requires about 44 percent more space 
when block sizes are not constrained to be powers of 2. 

A corresponding time estimate for the garbage collection and compacting 
algorithm of exercise 33 is about 104 units of time to locate a free node, assuming 
that garbage collection occurs when the memory is approximately half full, and 
assuming that the nodes have an average length of 5 words with 2 links per 
node. The pros and cons of garbage collection are discussed in Section 2.3.5. 
When the memory is not heavily loaded and when the appropriate restrictions 
are met, garbage collection and compacting is very efficient; for example, on the 
MIX computer, the garbage collection method is faster than the other two, if the 
accessible items never occupy more than about one-third of the total memory 
space, and if the nodes are relatively small. 

If the assumptions underlying garbage collection are met, the best strategy 
may be to divide the pool of memory into two halves and to do all allocation 
sequentially within one half. Instead of freeing blocks as they become available, 
we simply wait until the current active half of memory is full; then we can copy all 
active data to the other half, simultaneously removing all holes between blocks, 
with a method like that of exercise 33. The size of each half pool might also be 
adjusted as we switch from one half to the other. 


The simulation techniques mentioned above were applied also to some other 
storage allocation algorithms. The other methods were so poor by comparison 
with the algorithms of this section that they will be given only brief mention here: 

a) Separate AVAIL lists were kept for each size. A single free block was 
occasionally split into two smaller blocks when necessary, but no attempt was 
made to put such blocks together again. The memory map became fragmented 
into finer and finer parts until it was in terrible shape; a simple scheme like this 
is almost equivalent to doing separate allocation in disjoint areas, one area for 
each block size. 
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b) An attempt was made to do two-level allocation: The memory was divided 
into 32 large sectors. A brute-force allocation method was used to reserve large 
blocks of 1, 2, or 3 (rarely more) adjacent sectors; each large block such as this 
was subdivided to meet storage requests until no more room was left within 
the current large block, and then another large block was reserved for use in 
subsequent allocations. Each large block was returned to free storage only when 
all space within it became available. This method almost always ran out of 
storage space very quickly. 

Although this particular method of two-level allocation was a failure for 
the data considered in the author’s simulation experiments, there are other 
circumstances (which occur not infrequently in practice) when a multiple-level 
allocation strategy can be beneficial. For example, if a rather large program 
operates in several stages, we might know that certain types of nodes are needed 
only within a certain subroutine. Some programs might also find it desirable 
to use quite different allocation strategies for different classes of nodes. The 
idea of allocating storage by zones, with possibly different strategies employed 
in each zone and with the ability to free an entire zone at once, is discussed by 
Douglas T. Ross in CACM 10 (1967), 481-492. 


For further empirical results about dynamic storage allocation, see the arti- 
cles by B. Randell, CACM 12 (1969), 365-369, 372; P. W. Purdom, S. M. Stigler, 
and T. O. Cheam, BIT 11 (1971), 187-195; B. H. Margolin, R. P. Parmelee, and 
M. Schatzoff, IBM Systems J. 10 (1971), 283-304; J. A. Campbell, Comp. J. 
14 (1971), 7-9; John E. Shore, CACM 18 (1975), 433-440; Norman R. Nielsen, 
CACM 20 (1977), 864-873. 


*E. Distributed fit. If the distribution of block sizes is known in advance, 
and if each block present is equally likely to be the next one freed regardless 
of when it was allocated, we can use a technique that has substantially better 
memory utilization than the general-purpose techniques described so far, by 
following the suggestions of E. G. Coffman, Jr., and F. T. Leighton [J. Computer 
and System Sci. 38 (1989), 2-35]. Their “distributed-fit method” works by 
partitioning memory into roughly N + VN lg N slots, where N is the desired 
maximum number of blocks to be handled in steady state. Each slot has a fixed 
size, although different slots may have different sizes; the main point is that any 
given slot has fixed boundaries, and it will either be empty or contain a single 
allocated block. 

The first N slots in Coffman and Leighton’s scheme are laid out according 
to the assumed distribution of sizes, while the last VN lg N slots all have the 
maximum size. For example, if we assume that the block sizes will be uniformly 
distributed between 1 and 256, and if we expect to handle N = 214 such blocks, 
we would divide the memory into N/256 = 2° slots of each size 1, 2, ..., 256, 
followed by an “overflow area” that contains VN lg N = 27-14 = 1792 blocks of 
size 256. When the system is operating at full capacity, we expect it to handle 

257 257 


N blocks of average size =>", occupying ="N = 271 + 213 = 2,105,344 locations; 


this is the amount of space we have allocated to the first N slots. We have also 
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set aside an additional 1792 - 256 = 458,752 locations to handle the effects of 
random variations; this additional overhead amounts to O(N~!/? log N) of the 
total space, rather than a constant multiple of N as in the buddy system, so it 
becomes a negligible fraction when N — oo. In our example, however, it still 
amounts to about 18% of the total allocation. 

The slots should be arranged in order so that the smaller slots precede the 
larger ones. Given this arrangement, we can allocate blocks by using either the 
first-fit or the best-fit technique. (Both methods are equivalent in this case, 
because the slot sizes are ordered.) The effect, under our assumptions, is to 
start searching at an essentially random place among the first N slots whenever 
a new allocation request comes in, and to continue until we find an empty slot. 


If the starting slot for each search is truly random between 1 and N, we will 
not have to invade the overflow area very often. Indeed, if we insert exactly N 
items starting at random slots, overflow will occur only O(VN ) times, on the 
average. The reason is that we can compare this algorithm to hashing with linear 
probing (Algorithm 6.4L), which has the same behavior except that the search 
for an empty cell wraps around from N to 1 instead of going into an overflow 
area. The analysis of Algorithm 6.4L in Theorem 6.4K shows that, when N 
items have been inserted, the average displacement of each item from its hash 
address is $(Q(N) — 1) ~ \/aN/8; by circular symmetry this average is easily 
seen to be the same as the average number of times a search goes from slot k 
to slot k + 1, for each k. Overflows in the distributed-fit method correspond to 
searches that go from slot N to slot 1, except that our situation is even better 
because we avoid some congestion by not wrapping around. Therefore fewer than 
\/7N/8 overflows will occur, on the average. This analysis does not take account 
of deletions, which preserve the assumptions of Algorithm 6.4L only if we move 
blocks back when deleting another block that intervened between their starting 
slots and their allocated slots (see Algorithm 6.4R); again, however, moving 
them back would only increase the chance of overflow. Our analysis also fails to 
account for the effect of having more than N blocks present at once; this can 
happen if we assume only that the arrival time between blocks is about one Nth 
of the residence time. For the case of more than N blocks we need to extend the 
analysis of Algorithm 6.4L, but Coffman and Leighton proved that the overflow 
area will almost never need more than VN lg N slots; the probability of running 
off the end is less than O(N~™) for all M. 

In our example, the starting slot for the search during an allocation is not 
uniform among slots 1, 2,..., N; it is, instead, uniform among slots 1, 65, 129, 
..., N — 63, because there are N/256 = 64 slots of each size. But this deviation 
from the random model considered in the previous paragraph makes overflow 
even less likely than predicted. All bets are off, of course, if the assumptions 
about block size distribution and occupancy time are violated. 


F. Overflow. What do we do when no more room is available? Suppose there 
is a request for, say, n consecutive words, when all available blocks are too small. 
The first time this happens, there usually are more than n available locations 
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present, but they are not consecutive; compacting memory (that is, moving 
some of the locations that are in use, so that all available locations are brought 
together) would mean that we could continue processing. But compacting is slow, 
and it requires a disciplined use of pointers; moreover, the vast majority of cases 
in which the first-fit method runs out of room will soon thereafter run completely 
out of space anyway, no matter how much compacting and re-compacting is 
done. Therefore it is generally not worthwhile to write a compacting program, 
except under special circumstances in connection with garbage collection, as 
in exercise 33. If overflow is expected to occur, some method for removing 
items from memory and storing them on an external memory device can be 
used, with provision for bringing the information back again when it is needed. 
This implies that all programs referring to the dynamic memory area must be 
severely restricted with regard to the allowable references they make to other 
blocks, and special computer hardware (for example, interrupt on absence of 
data, or automatic “paging”) is generally required for efficient operation under 
these conditions. 

Some decision procedure is necessary to decide which blocks are the most 
likely candidates for removal. One idea is to maintain a doubly linked list of the 
reserved blocks, in which a block is moved up to the front of the list each time 
it is accessed; then the blocks are effectively sorted in order of their last access, 
and the block at the rear of the list is the one to remove first. A similar effect 
can be achieved more simply by putting the reserved blocks into a circular list 
and including a “recently used” bit in each block; the latter is set to 1 whenever 
the block is accessed. When it is time to remove a block, a pointer moves along 
the circular list, resetting all “recently used” bits to 0 until finding a block that 
has not been used since the last time the pointer reached this part of the circle. 

J. M. Robson has shown [JACM 18 (1971), 416-423] that dynamic storage 
allocation strategies that never relocate reserved blocks cannot possibly be guar- 
anteed to use memory efficiently; there will always be pathological circumstances 
in which the method breaks down. For example, even when blocks are restricted 
to be of sizes 1 and 2, overflow might occur with the memory only about 2 full, 
no matter what allocation algorithm is used! Robson’s interesting results are 
surveyed in exercises 36—40, and in exercises 42—43 where he has shown that the 
best-fit method has a very bad worst case by comparison with first-fit. 


G. For further reading. A comprehensive survey and critical review of 
dynamic storage allocation techniques, based on many more years of experience 
than were available to the author when the material above was written, has 
been compiled by Paul R. Wilson, Mark S. Johnstone, Michael Neely, and David 
Boles, Lecture Notes in Computer Science 986 (1995), 1-116. 


EXERCISES 


1. [20] What simplifications can be made to the reservation and liberation algo- 
rithms of this section, if storage requests always appear in a “last-in-first-out” manner, 
that is, if no reserved block is freed until after all blocks that were reserved subsequently 
have already been freed? 
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2. [HM23] (E. Wolman.) Suppose that we want to choose a fixed node size for 
variable-length items, and suppose also that when each node has length k and when an 
item has length J, it takes [1/(k — b)] nodes to store this item. (Here b is a constant, 
signifying that b words of each node contain control information, such as a link to 
the next node.) If the average length l of an item is L, what choice of k minimizes 
the average amount of storage space required? (Assume that the average value of 
(1/(k — b)) mod 1 is equal to 1/2, for any fixed k, as l varies.) 


3. [40] By computer simulation, compare the best-fit, first-fit, and worst-fit methods 
of storage allocation; in the latter method, the largest available block is always chosen. 
Is there any significant difference in the memory usage? 


4. [22] Write a MIX program for Algorithm A, paying special attention to making 
the inner loop fast. Assume that the SIZE field is (4:5), the LINK field is (0:2), and 
A<0. 


5. [18] Suppose it is known that N is always 100 or more in Algorithm A. Would it 
be a good idea to set c = 100 in the modified step A4’? 


6. [23] (Nezt fit.) After Algorithm A has been used repeatedly, there will be a strong 
tendency for blocks of small SIZE to remain at the front of the AVAIL list, so that it will 
often be necessary to search quite far into the list before finding a block of length N or 
more. For example, notice how the size of the blocks essentially increases in Fig. 42, 
for both reserved and free blocks, from the beginning of memory to the end. (The 
AVAIL list used while Fig. 42 was being prepared was kept sorted by order of location, 
as required by Algorithm B.) Can you suggest a way to modify Algorithm A so that 
(a) short blocks won’t tend to accumulate in a particular area, and (b) the AVAIL list 
may still be kept in order of increasing memory locations, for purposes of algorithms 
like Algorithm B? 


7. [10] The example (1) shows that first-fit can sometimes be definitely superior to 
best-fit. Give a similar example that shows a case where best-fit is superior to first-fit. 


8. [21] Show how to modify Algorithm A in a simple way to obtain an algorithm for 
the best-fit method, instead of first-fit. 


9. [26] In what ways could a reservation algorithm be designed to use the best-fit 
method, without searching through the whole AVAIL list? (Try to think of ways that 
cut down the necessary search as much as possible.) 


10. [22] Show how to modify Algorithm B so that the block of N consecutive cells 
beginning at location PO is made available, without assuming that each of these N cells 
is currently unavailable; assume, in fact, that the area being freed may actually overlap 
several blocks that are already free. 


11. [M25] Show that the improvement to Algorithm A suggested in the answer to 
exercise 6 can also be used to lead to a slight improvement in Algorithm B, which cuts 
the average length of search from half the length of the AVAIL list to one-third this 
length. (Assume that the block being freed will be inserted into a random place within 
the sorted AVAIL list.) 


12. [20] Modify Algorithm A so that it follows the boundary-tag conventions of 
(7)-(9), uses the modified step A4’ described in the text, and also incorporates the 
improvement of exercise 6. 


13. [21] Write a MIX program for the algorithm of exercise 12. 
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14. [21] What difference would it make to Algorithm C and the algorithm of exer- 
cise 12, (a) if the SIZE field were not present in the last word of a free block? or (b) if 
the SIZE field were not present in the first word of a reserved block? 

15. [24] Show how to speed up Algorithm C at the expense of a slightly longer 
program, by not changing any more links than absolutely necessary in each of four 
cases depending on whether TAG(PO — 1), TAG(PO + SIZE(PO)) are plus or minus. 


16. [24] Write a MIX program for Algorithm C, incorporating the ideas of exercise 15. 


17. [10] What should the contents of LOC(AVAIL) and LOC(AVAIL) +1 be in (g) when 
there are no available blocks present? 


18. [20] Figures 42 and 43 were obtained using the same data, and essentially the 
same algorithms (Algorithms A and B), except that Fig. 43 was prepared by modifying 
Algorithm A to choose best-fit instead of first-fit. Why did this cause Fig. 42 to have a 
large available area in the higher locations of memory, while in Fig. 43 there is a large 
available area in the lower locations? 


19. [24] Suppose that blocks of memory have the form of (7), but without the TAG or 
SIZE fields required in the last word of the block. Suppose further that the following 
simple algorithm is being used to make a reserved block free again: Q + AVAIL, 
LINK(PO) + Q, LINK(PO +1) «+ LOC(AVAIL), LINK(Q+1) « PO, AVAIL «+ PO, TAG(PO) + 
“_” (This algorithm does nothing about collapsing adjacent areas together.) 

Design a reservation algorithm, similar to Algorithm A, that does the necessary 
collapsing of adjacent free blocks while searching the AVAIL list, and at the same time 
avoids any unnecessary fragmentation of memory as in (2), (3), and (4). 


20. [00] Why is it desirable to have the AVAIL[k] lists in the buddy system doubly 
linked instead of simply having straight linear lists? 


21. [HM25| Examine the ratio an/bn, where an is the sum of the first n terms of 
14+24+44+4+4+8+4+8+4+8+4+8+416+16+---, and bn is the sum of the first n terms of 
1+2+3+4+5+6+7+8+9+10+---, asn goes to infinity. 


22. [21] The text repeatedly states that the buddy system allows only blocks of size 2* 
to be used, and exercise 21 shows this can lead to a substantial increase in the storage 
required. But if an 11-word block is needed in connection with the buddy system, why 
couldn’t we find a 16-word block and divide it into an 11-word piece together with two 
free blocks of sizes 4 and 1? 


23. [05] What is the binary address of the buddy of the block of size 4 whose binary 
address is 011011110000? What would it be if the block were of size 16 instead of 4? 
24. [20] According to the algorithm in the text, the largest block (of size 2”) has no 


buddy, since it represents all of storage. Would it be correct to define buddy,,,(0) = 0 
(namely, to make this block its own buddy), and then to avoid testing k = m in step S1? 


25. [22] Criticize the following idea: “Dynamic storage allocation using the buddy 
system will never reserve a block of size 2” in practical situations (since this would fill 
the whole memory), and, in general, there is a maximum size 2” for which no blocks of 
greater size will ever be reserved. Therefore it is a waste of time to start with such large 
blocks available, and to combine buddies in Algorithm S when the combined block has 
a size larger than 2”.” 


26. [21] Explain how the buddy system could be used for dynamic storage allocation 
in memory locations 0 through M— 1 even when M does not have the form 2™ as required 
in the text. 
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27. [24] Write a MIX program for Algorithm R, and determine its running time. 


28. [25] Assume that MIX is a binary computer, with a new operation code XOR defined 

as follows (using the notation of Section 1.3.1): “C =5, F =5. For each bit position in 

location M that equals 1, the corresponding bit position in register A is complemented 

(changed from 0 to 1 or 1 to 0); the sign of rA is unaffected. The execution time is 2u.” 
Write a MIX program for Algorithm S, and determine its running time. 


29. [20] Could the buddy system do without the tag bit in each reserved block? 


30. [M48] Analyze the average behavior of Algorithms R and S, given reasonable 
distributions for the sequence of storage requests. 


31. [M40] Can a storage allocation system analogous to the buddy system be designed 
using the Fibonacci sequence instead of powers of two? (Thus, we might start with Fm 
available words, and split an available block of Fẹ words into two buddies of respective 
lengths F,-1 and Fp—2.) 

32. [HM46] Determine limn+oo Qn, if it exists, where a» is the mean value of tn in 
a random sequence defined as follows: Given the values of tk for 0 < k < n, let tn be 
chosen uniformly from {1,2,..., gn}, where 


In = | Ž min(10000, f(tn—1 = 1), f(tn—2 a 2), SA3 F(to ~~ n))|, 


and f(x) = z if x > 0, f(z) =o if x <0. [Note: Some limited empirical tests indicate 
that an might be approximately 14, but this is probably not very accurate.] 


33. [28] (Garbage collection and compacting.) Assume that memory locations 1, 2, 
..., AVAIL — 1 are being used as a storage pool for nodes of varying sizes, having the 
following form: The first word of NODE(P) contains the fields 


SIZE(P) = number of words in NODE(P); 
T(P) = number of link fields in NODE(P); T(P) < SIZE(P); 
LINK(P) = special link field for use only during garbage collection. 


The node immediately following NODE(P) in memory is NODE(P + SIZE(P)). Assume 
that the only fields in NODE(P) that are used as links to other nodes are LINK(P + 1), 
LINK(P + 2), ..., LINK(P + T(P)), and each of these link fields is either A or the address 
of the first word of another node. Finally, assume that there is one further link variable 
in the program, called USE, and it points to one of the nodes. 

Design an algorithm that (i) determines all nodes accessible directly or indirectly 
from the variable USE, (ii) moves these nodes into memory locations 1 through K— 1, for 
some K, changing all links so that structural relationships are preserved, and (iii) sets 
AVAIL + K. 

For example, consider the following contents of memory, where INFO(L) denotes 
the contents of location L, excluding LINK(L): 


1: SIZE=2,T=1 6: SIZE=2,T=0 AVAIL = 11, 
2: LINK = 6, INFO = A 7: CONTENTS = D USE = 3. 
3: SIZE=3,T=1 8: SIZE = 3,T=2 
4: LINK = 8, INFO = B 9: LINK = 8, INFO = E 
5: CONTENTS = C 10: LINK = 3, INFO = F 
Your algorithm should transform this into 
1: SIZE= 3, T=1 4: SIZE= 3, T=2 AVAIL = 7, 
2: LINK = 4, INFO = B 5: LINK = 4, INFO = E USE = 1. 


3: CONTENTS = C 6: LINK = 1, INFO = F 
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34. [29] Write a MIX program for the algorithm of exercise 33, and determine its 
running time. 


35. [22] Contrast the dynamic storage allocation methods of this section with the 
techniques for variable-size sequential lists discussed at the end of Section 2.2.2. 


36. [20] A certain lunch counter in Hollywood, California, contains 23 seats in a row. 
Diners enter the shop in groups of one or two, and a glamorous hostess shows them 
where to sit. Prove that she will always be able to seat people immediately without 
splitting up any pairs, if no customer who comes alone is assigned to any of the seats 
numbered 2, 5, 8, ..., 20, provided that there never are more than 16 customers present 
at a time. (Pairs leave together.) 


37. [26] Continuing exercise 36, prove that the hostess can’t always do such a good 
job when there are only 22 seats at the counter: No matter what strategy she uses, 
it will be possible to reach a situation where two friends enter and only 14 people are 
seated, but no two adjacent seats are vacant. 


38. [M21] (J. M. Robson.) The lunch-counter problem in exercises 36 and 37 can be 
generalized to establish the worst-case performance of any dynamic storage allocation 
algorithm that never relocates reserved blocks. Let N(n,m) be the smallest amount of 
memory such that any series of requests for allocation and liberation can be handled 
without overflow, provided that all block sizes are < m and the total amount of space 
requested never exceeds n. Exercises 36 and 37 prove that N(16,2) = 23; determine 
the exact value of N(n, 2) for all n. 


39. [HM23] (J. M. Robson.) In the notation of exercise 38, show that N(n1+n2, m) < 
N(n1, m)+N (n2, m)+N(2m—2, m); hence for fixed m, limno N(n,m)/n = N(m) 
exists. 

40. [HM50] Continuing exercise 39, determine N(3), N(4), and limm— oo N(m)/lgm 
if it exists. 

41. [M27] The purpose of this exercise is to consider the worst-case memory usage 
of the buddy system. A particularly bad case occurs, for example, if we start with an 
empty memory and proceed as follows: First reserve n = 2"'' blocks of length 1, which 
go into locations 0 through n — 1; then for k = 1, 2, ..., r, liberate all blocks whose 
starting location is not divisible by 2”, and reserve 2~*~'n blocks of length 2%, which 
go into locations į(1 + k)n through $(2+k)n—1. This procedure uses 1 + $r times 
as much memory as is ever occupied. 

Prove that the worst case cannot be substantially worse than this: When all 
requests are for block sizes 1, 2, ..., 2”, and if the total space requested at any time 
never exceeds n, where n is a multiple of 2", the buddy system will never overflow a 
memory area of size (r + 1)n. 


42. [M40] (J. M. Robson, 1975.) Let Ngr(n,m) be the amount of memory needed to 
guarantee non-overflow when the best-fit method is used for allocation as in exercise 38. 
Find an attacking strategy to show that Ngr(n,m) > mn — O(n + m°). 

43. [HM35] Continuing exercise 42, let Nrr(n,m) be the memory needed when the 
first-fit method is used. Find a defensive strategy to show that Nrr(n,m) < Hmn/In 2. 
(Hence the worst case of first-fit is not far from the best possible worst case.) 

44. [M21] Suppose the distribution function F(x) = (probability that a block has 
size < x) is continuous. For example, F(x) is (x — a)/(b— a) for a < x < b if the sizes 
are uniformly distributed between a and b. Give a formula that expresses the sizes of 
the first N slots that should be set up when we use the distributed-fit method. 
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2.6. HISTORY AND BIBLIOGRAPHY 


LINEAR LISTS and rectangular arrays of information kept in consecutive memory 
locations were widely used from the earliest days of stored-program computers, 
and the earliest treatises on programming gave the basic algorithms for traversing 
these structures. [For example, see J. von Neumann, Collected Works 5, 113- 
116 (written 1946); M. V. Wilkes, D. J. Wheeler, S. Gill, The Preparation 
of Programs for an Electronic Digital Computer (Reading, Mass.: Addison- 
Wesley, 1951), subroutine V-1; and see especially also the work of Konrad Zuse, 
Berichte der Gesellschaft fiir Mathematik und Datenverarbeitung 63 (Bonn: 
1972), written in 1945. Zuse was the first to develop nontrivial algorithms 
that worked with lists of dynamically varying lengths.] Before the days of 
index registers, operations on sequential linear lists were done by performing 
arithmetic on the machine language instructions themselves, and the need to do 
such arithmetic was one of the early motivations for having a computer whose 
programs share memory space with the data they manipulate. 

Techniques that permit variable-length linear lists to share sequential loca- 
tions, in such a way that they shift back and forth when necessary as described 
in Section 2.2.2, were apparently a much later invention. J. Dunlap of Digitek 
Corporation developed such techniques before 1963 in connection with the de- 
sign of a series of compiler programs; about the same time the idea appeared 
independently in the design of a COBOL compiler at IBM Corporation, and a 
collection of related subroutines called CITRUS was subsequently used at various 
installations. The techniques remained unpublished until after they had been 
developed independently by Jan Garwick of Norway; see BIT 4 (1964), 137-140. 

The idea of having linear lists in nonsequential locations seems to have orig- 
inated in connection with the design of computers with rotating drum memories. 
After executing the instruction in location n, such a computer was usually not 
ready to get its next instruction from location n + 1, because the drum had 
already rotated past that point. Depending on the instruction being performed, 
the most favorable position for the next instruction might be n+ 7 or n+ 18, 
say, and the machine could operate up to six or seven times faster if its in- 
structions were located optimally rather than consecutively. [For a discussion of 
the interesting problems concerning the best placement of instructions, see the 
author’s article in JACM 8 (1961), 119-150.] Therefore an extra address field 
was provided in each machine language instruction, to serve as a link to the next 
command. This idea, called “one-plus-one addressing,” was discussed by John 
Mauchly in 1946 [Theory and Techniques for the Design of Electronic Comput- 
ers 4 (U. of Pennsylvania, 1946), Lecture 37]; it contained the notion of linked 
lists in embryonic form, although the dynamic insertion and deletion operations 
that we have used so frequently in this chapter were still unknown. Another 
early appearance of links in programs was in H. P. Luhn’s 1953 memorandum 
suggesting the use of “chaining” for external searching; see Section 6.4. 

Linked memory techniques were really born when A. Newell, J. C. Shaw, 
and H. A. Simon began their investigations of heuristic problem-solving by 
machine. As an aid to writing programs that searched for proofs in mathematical 
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logic, they designed the first List-processing language, IPL-II, in the spring of 
1956. (IPL was an acronym for Information Processing Language.) This was 
a system that made use of pointers and included important concepts like the 
list of available space, but the concept of stacks was not yet well developed. 
IPL-III, designed a year later, included “push down” and “pop up” for stacks as 
important basic operations. [For references to IPL-II see IRE Transactions IT-2 
(September 1956), 61-70; Proc. Western Joint Comp. Conf. 9 (1957), 218-240. 
Material on IPL-III first appeared in course notes given at the University of 
Michigan in the summer of 1957.] 

The work of Newell, Shaw, and Simon inspired many other people to use 
linked memory, which was often referred to as NSS memory at the time, but 
mostly for problems dealing with simulation of human thought processes. Grad- 
ually, the techniques became recognized as basic computer-programming tools; 
the first article describing the usefulness of linked memory for “down-to-earth” 
problems was published by J. W. Carr, III, in CACM 2,2 (February 1959), 4-6. 
Carr pointed out in this article that linked lists can readily be manipulated in 
ordinary programming languages, without requiring sophisticated subroutines 
or interpretive systems. See also G. A. Blaauw, “Indexing and control-word 
techniques,” IBM J. Res. and Dev. 3 (1959), 288-301. 

At first, one-word nodes were used for linked tables, but about 1959 the 
usefulness of several consecutive words per node and “multilinked” lists was 
gradually being discovered by several different groups of people. The first article 
dealing specifically with this idea was published by D. T. Ross, CACM 4 (1961), 
147-150. At that time he used the term “plex” for what has been called a “node” 
in this chapter, but he subsequently used the word “plex” in a different sense to 
denote a class of nodes combined with associated algorithms for their traversal. 

Notations for referring to fields within nodes are generally of two kinds: The 
name of the field either precedes or follows the pointer designation. Thus, while 
we have written “INFO(P)” in this chapter, some other authors write, for exam- 
ple, “P. INFO”. At the time this chapter was prepared, the two notations seemed 
to be equally prominent. The notation adopted here has the great advantage 
that it translates immediately into FORTRAN, COBOL, or similar languages, if 
we define INFO and LINK arrays and use P as the index. Furthermore it seems 
natural to use mathematical functional notation to describe attributes of a node. 
Note that “INFO(P)” is pronounced “info of P” in conventional mathematical 
verbalization, just as f(x) is rendered “f of x.” The alternative notation P. INFO 
has less of a natural flavor, since it tends to put the emphasis on P, although it 
can be read “P’s info”; the reason INFO(P) seems preferable is apparently the 
fact that P is variable, but INFO has a fixed significance when the notation is 
employed. By analogy, we could consider a vector A = (A[l1], A[2],..., A[100]) 
to be a node having 100 fields named 1,2,...,100. Now the second field would 
be referred to as “2(P)” in our notation, where P points to the vector A; but 
if we are referring to the jth element of the vector, we find it more natural to 
write A[j], putting the variable quantity “j” second. Similarly it seems most 
appropriate to put the variable quantity “P” second in the notation INFO(P). 
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Perhaps the first people to recognize that the concepts “stack” (last-in-first- 
out) and “queue” (first-in-first-out) are important objects of study were cost 
accountants interested in reducing income tax assessments; for a discussion of 
the “LIFO” and “FIFO” methods of pricing inventories, see any intermediate ac- 
counting textbook, e.g., C. F. and W. J. Schlatter, Cost Accounting (New York: 
Wiley, 1957), Chapter 7. In the mid-1940s, A. M. Turing developed a stack mech- 
anism called Reversion Storage for subroutine linkage, local variables, and pa- 
rameters. His names for “push” and “pop” were “bury” and “disinter/unbury.” 
(See the references in Section 1.4.5.) No doubt simple uses of stacks kept in 
sequential memory locations were common in computer programming from the 
earliest days, since a stack is such an intuitive concept. The programming of 
stacks in linked form appeared first in IPL, as stated above; the name “stack” 
stems from IPL terminology (although “pushdown list” was the more official IPL 
wording), and it was also independently introduced by E. W. Dijkstra [Numer. 
Math. 2 (1960), 312-318]. “Deque” is a term coined by E. J. Schweppe in 1966. 

The origin of circular and doubly linked lists is obscure; presumably these 
ideas occurred naturally to many people. A strong factor in the popularization 
of such techniques was the existence of general List-processing systems based on 
them [principally the Knotted List Structures, CACM 5 (1962), 161-165, and 
Symmetric List Processor, CACM 6 (1963), 524-544, of J. Weizenbaum]. Ivan 
Sutherland introduced the use of independent doubly linked lists within larger 
nodes, in his Sketchpad system (Ph.D. thesis, Mass. Inst. of Technology, 1963). 

Various methods for addressing and traversing multidimensional arrays of 
information were developed independently by clever programmers since the ear- 
liest days of computers, and thus another part of the unpublished computer 
folklore was born. This subject was first surveyed in print by H. Hellerman, 
CACM 5 (1962), 205-207. See also J. C. Gower, Comp. J. 4 (1962), 280-286. 

Tree structures represented explicitly in computer memory were originally 
used for applications to algebraic formula manipulation. The machine language 
for several early computers used a three-address code to represent the compu- 
tation of arithmetic expressions; the latter is equivalent to the INFO, LLINK, 
and RLINK of a binary tree representation. In 1952, H. G. Kahrimanian devel- 
oped algorithms for differentiating algebraic formulas represented in an extended 
three-address code; see Symposium on Automatic Programming (Washington, 
D.C.: Office of Naval Research, May 1954), 6-14. 

Since then, tree structures in various guises have been studied independently 
by many people in connection with numerous computer applications, but the 
basic techniques for tree manipulation (not general List manipulation) have 
seldom appeared in print except in detailed description of particular algorithms. 
The first general survey was made in connection with a more general study of 
all data structures by K. E. Iverson and L. R. Johnson [IBM Corp. research 
reports RC-390, RC-603, 1961; see Iverson, A Programming Language (New 
York: Wiley, 1962), Chapter 3]. See also G. Salton, CACM 5 (1962), 103-114. 

The concept of threaded trees is due to A. J. Perlis and C. Thornton, CACM 
3 (1960), 195-204. Their paper also introduced the important idea of traversing 
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trees in various orders, and gave numerous examples of algebraic manipulation 
algorithms. Unfortunately, this important paper was prepared hastily and it 
contains many misprints. The threaded lists of Perlis and Thornton were only 
“right-threaded trees” in our terminology; binary trees that are threaded in 
both directions were independently discovered by A. W. Holt, A Mathematical 
and Applied Investigation of Tree Structures (Thesis, U. of Pennsylvania, 1963). 
Postorder and preorder for the nodes of trees were called “normal along order” 
and “dual along order” by Z. Pawlak [Colloquium on the Foundation of Math- 
ematics, Tihany, 1962 (Budapest: Akadémiai Kiadó, 1965), 227-238]. Preorder 
was called “subtree order” by Iverson and Johnson in the references cited above. 
Graphical ways to represent the connection between tree structures and cor- 
responding linear notations were described by A. G. Oettinger, Proc. Harvard 
Symp. on Digital Computers and their Applications (April 1961), 203-224. The 
representation of trees in preorder by degrees, with associated algorithms relating 
this representation to Dewey decimal notation and other properties of trees, was 
presented by S. Gorn, Proc. Symp. Math. Theory of Automata (Brooklyn: Poly. 
Inst., 1962), 223-240. 

The history of tree structures as mathematical entities, together with a 
bibliography of the subject, is reviewed in Section 2.3.4.6. 

At the time this section was first written in 1966, the most widespread 
knowledge about information structures was due to programmers’ exposure to 
List processing systems, which played a very important part in this history. 
The first widely used system was IPL-V (a descendant of IPL-III, developed late 
in 1959); IPL-V was an interpretive system in which a programmer learned a 
machine-like language for List operations. At about the same time, FLPL (a set 
of FORTRAN subroutines for List manipulation, also inspired by IPL but using 
subroutine calls instead of interpretive language) was developed by H. Gelernter 
and others. A third system, LISP, was designed by J. McCarthy, also in 1959. 
LISP was quite different from its predecessors: Its programs were (and still 
are) expressed in mathematical functional notation combined with “conditional 
expressions,” then converted into a List representation. Many List processing 
systems came into existence during the 1960s; the most prominent among these 
from a historical standpoint was J. Weizenbaum’s SLIP, a set of subroutines that 
implemented doubly linked Lists in FORTRAN. 

An article by Bobrow and Raphael, CACM 7 (1964), 231-240, may be 
read as a brief introduction to IPL-V, LISP, and SLIP; it gives a comparison 
of these systems. An excellent early introduction to LISP was published by 
P. M. Woodward and D. P. Jenkins, Comp. J. 4 (1961), 47-53. See also the 
authors’ discussions of their own systems, which are articles of considerable 
historical importance: “An introduction to IPL-V” by A. Newell and F. M. Tonge, 
CACM 3 (1960), 205-211; “A FORTRAN-compiled List Processing Language” 
by H. Gelernter, J. R. Hansen, and C. L. Gerberich, JACM 7 (1960), 87- 
101; “Recursive functions of symbolic expressions and their computation by 
machine, I” by John McCarthy, CACM 3 (1960), 184-195; “Symmetric List 
Processor” by J. Weizenbaum, CACM 6 (1963), 524-544. Weizenbaum’s article 
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included a complete description of all of the algorithms used in SLIP. Of all these 
early systems, only LISP had the necessary ingredients to survive the ensuing 
decades of further progress. McCarthy has described LISP’s early history in 
History of Programming Languages (Academic Press, 1981), 173-197. 

Several string manipulation systems also appeared during the 1960s; they 
were primarily concerned with operations on variable-length strings of alphabetic 
information — looking for occurrences of certain substrings and replacing them 
by others, etc. The most important of these from a historical perspective were 
COMIT [V. H. Yngve, CACM 6 (1963), 83-84] and SNOBOL [D. J. Farber, R. 
E. Griswold, and I. P. Polonsky, JACM 11 (1964), 21-30]. String manipulation 
systems were used widely, and they were composed primarily of algorithms like 
the ones we have seen in this chapter, but they played a comparatively small 
role in the history of the techniques of information structure representation; users 
of such systems were isolated from the details of the actual internal processes 
carried on by the computer. For a survey of early string manipulation techniques, 
see S. E. Madnick, CACM 10 (1967), 420-424. 

The IPL-V and FLPL systems for List-processing did not use either a garbage 
collection or a reference count technique for the problem of shared Lists; instead, 
each List was “owned” by one List and “borrowed” by all other Lists that referred 
to it, and a List was erased when its “owner” allowed it to disappear. Hence, the 
programmer was enjoined to make sure that no List was still borrowing any Lists 
that were being erased. The reference counter technique for Lists was introduced 
by G. E. Collins, CACM 3 (1960), 655-657, and explained further in CACM 9 
(1966), 578-588. Garbage collection was first described in McCarthy’s article of 
1960; see also Weizenbaum’s remarks in CACM 7 (1964), 38, and an article by 
Cohen and Trilling, BIT 7 (1967), 22-30. 

An increasing realization of the importance of link manipulations led natu- 
rally to their inclusion in algebraic programming languages designed after 1965. 
The new languages allowed programmers to choose suitable forms of data rep- 
resentation without resorting to assembly language or paying the overhead of 
completely general List structures. Some of the fundamental steps in this de- 
velopment were the work of N. Wirth and H. Weber [CACM 9 (1966), 13-23, 
25, 89-99]; H. W. Lawson [CACM 10 (1967), 358-367]; C. A. R. Hoare [Symbol 
Manipulation Languages and Techniques, ed. by D. G. Bobrow (Amsterdam: 
North-Holland, 1968), 262-284]; O.-J. Dahl and K. Nygaard [CACM 9 (1966), 
671-678]; A. van Wijngaarden, B. J. Mailloux, J. E. L. Peck, and C. H. A. 
Koster [Numerische Math. 14 (1969), 79-218]; Dennis M. Ritchie [History of 
Programming Languages —II (ACM Press, 1996), 671-698]. 

Dynamic storage allocation algorithms were in use several years before they 
were ever described in print. A very readable discussion was prepared by W. 
T. Comfort in 1961 and published in CACM 7 (1964), 357-362. The boundary- 
tag method, introduced in Section 2.5, was designed by the author in 1962 for 
use in an operating system for the Burroughs B5000 computer. The buddy 
system was first used by H. Markowitz in connection with the SIMSCRIPT 
programming system in 1963, and it was independently discovered and published 
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by K. Knowlton, CACM 8 (1965), 623-625; see also CACM 9 (1966), 616-625. 
For additional early discussions of dynamic storage allocation, see the articles 
by Iliffe and Jodeit, Comp. J. 5 (1962), 200-209; Bailey, Barnett, and Burleson, 
CACM 7 (1964), 339-346; A. T. Berztiss, CACM 8 (1965), 512-513; and D. T. 
Ross, CACM 10 (1967), 481-492. 

A general discussion of information structures and their relation to program- 
ming was prepared by Mary d’Imperio, “Data Structures and their Representa- 
tion in Storage,” Annual Review in Automatic Programming 5 (Oxford: Perga- 
mon Press, 1969). Her paper is a valuable guide to the history of the topic, since 
it includes a detailed analysis of the structures used in connection with twelve 
List processing and string manipulation systems. See also the proceedings of two 
symposia, CACM 3 (1960), 183-234 and CACM 9 (1966), 567-643, for further 
historical details. (Several of the individual papers from those proceedings have 
already been cited above.) 

An excellent annotated bibliography of early work on symbol manipulation 
and algebraic formula manipulation, having numerous connections with the ma- 
terial of this chapter, was compiled by Jean E. Sammet; see Computing Reviews 
7 (July-August 1966), B1-B31. 


In this chapter we have looked at particular types of information structures 
in great detail, and (lest we fail to see the forest for the trees) it is perhaps 
wise to take stock of what we have learned and to summarize briefly the general 
subject of information structures from a broader perspective. Starting with the 
basic idea of a node as an element of data, we have seen many examples that 
illustrate convenient ways to represent structural relationships either implicitly 
(based on the relative order in which nodes are stored in computer memory) or 
explicitly (by means of links in the nodes, which point to other nodes). The 
amount of structural information that ought to be represented within the tables 
of a computer program depends on the operations that are to be performed on 
the nodes. 

For pedagogic reasons, we have largely concentrated on the connections 
between information structures and their machine representations, instead of 
discussing those issues separately. However, to gain a deeper understanding it is 
helpful to consider the subject from a more abstract point of view, distilling off 
several layers of ideas that can be studied by themselves. Several noteworthy ap- 
proaches of this kind have been developed, and the following thought-provoking 
papers are especially recommended from the early literature: G. Mealy, “Another 
look at data,” Proc. AFIPS Fall Joint Computer Conf. 31 (1967), 525-534; 
J. Earley, “Toward an understanding of data structures,” CACM 14 (1971), 617- 
627; C. A. R. Hoare, “Notes on data structuring,” in Structured Programming 
by O.-J. Dahl, E. W. Dijkstra, and C. A. R. Hoare (Academic Press, 1972), 83- 
174; Robert W. Engles, “A tutorial on data-base organization,” Annual Review 
in Automatic Programming 7 (1972), 3-63. 

The discussion in this chapter does not cover the entire subject of informa- 
tion structures in full generality; at least three important aspects of the subject 
have not been treated here: 
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a) We often want to search through a table to find a node or set of nodes 
possessing a certain value, and the need for such an operation often has a 
profound effect on the structure of the table. This situation is explored in detail 
in Chapter 6. 


b) We have primarily been concerned with the internal representation of 
structure within a computer; but that is obviously only part of the story, since 
structure must also be represented in the external input and output data. In 
simple cases, external structure can be treated by essentially the same techniques 
that we have been considering; but the processes of converting between strings of 
characters and more complex structures are also very important. Those processes 
are analyzed in Chapters 9 and 10. 


c) We have primarily discussed representations of structures within a high- 
speed random-access memory. When slower memory devices such as disks or 
tapes are being used, we find that all of the structural problems are intensified; 
it becomes much more crucial to have efficient algorithms and efficient schemes 
for data representation. Nodes that link to each other in such cases ought to 
go into nearby areas of the memory. Usually the problems are highly dependent 
on the characteristics of individual machines, so it is difficult to discuss them in 
general. The simpler examples treated in this chapter should help to prepare the 
reader for solving the more difficult problems that arise in connection with less 
ideal memory devices; Chapters 5 and 6 discuss some of these problems in detail. 


What are the main implications of the subjects treated in this chapter? 
Perhaps the most important conclusion we can reach is that the ideas we have 
encountered are not limited to computer programming alone; they apply more 
generally to everyday life. A collection of nodes containing fields, some of which 
point to other nodes, appears to be a very good abstract model for structural 
relationships of all kinds. This model shows how we can build up complicated 
structures from simple ones, and we have seen that corresponding algorithms for 
manipulating the structure can be designed in a natural manner. 

Therefore it seems appropriate to develop much more theory about linked 
sets of nodes than we know at this time. Perhaps the most obvious way to 
start such a theory is to define a new kind of abstract machine or “automaton” 
that deals with linked structures. For example, such a device might be defined 
informally as follows: There are numbers k, l, r, and s, such that the automaton 
processes nodes containing k link fields and r information fields; it has l link 
registers and s information registers, which enable it to control the processes 
it is performing. The information fields and registers may contain any symbols 
from some given set of information symbols; each of the link fields and link 
registers either contains A or points to a node. The machine can (i) create 
new nodes (putting a link to the node into a register), (ii) compare information 
symbols or link values for equality, and (iii) transfer information symbols or link 
values between registers and nodes. Only nodes pointed to by link registers 
are immediately accessible. Suitable restrictions on the machine’s behavior will 
make it equivalent to several other species of automata. 
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A related model of computation was proposed by A. N. Kolmogorov as 
early as 1952. His machine essentially operates on graphs G, having a specially 
designated starting vertex vo. The action at each step depends only on the 
subgraph G” consisting of all vertices at distance < n from vo in G, replacing 
G’ in G by another graph G” = f(G’), where G” includes vo and the vertices 
at distance exactly n from vo, and possibly other vertices (which are newly 
created); the remainder of graph G is left unaltered. Here n is a fixed number 
specified in advance for any particular algorithm, but it can be arbitrarily large. 
A symbol from a finite alphabet is attached to each vertex, and restrictions 
are made so that no two vertices with the same symbol can be adjacent to a 
common vertex. (See A. N. Kolmogorov, Uspekhi Mat. Nauk 8,4 (1953), 175- 
176; Kolmogorov and Uspensky, Uspekhi Mat. Nauk 13,4 (1958), 3-28; Amer. 
Math. Soc. Translations, series 2, 29 (1963), 217-245.) 

Linking automata can easily simulate graph machines, taking at most a 
bounded number of steps per graph step. Conversely, however, it is unlikely that 
graph machines can simulate arbitrary linking automata without unboundedly 
increasing the running time, unless the definition is changed from undirected to 
directed graphs, in view of the restriction to vertices of bounded degree. The 
linking model is, of course, quite close to the operations available to programmers 
on real machines, while the graph model is not. 

Some of the most interesting problems to solve for such devices would be 
to determine how fast they can solve certain problems, or how many nodes 
they need to solve certain problems (for example, to translate certain formal 
languages). At the time this chapter was first written, several interesting results 
of this kind had been obtained (notably by J. Hartmanis and R. E. Stearns) but 
only for special classes of Turing machines having multiple tapes and read/write 
heads. The Turing machine model is comparatively unrealistic, so these results 
tended to have little to do with practical problems. 

We must admit that, as the number n of nodes created by a linking automa- 
ton approaches infinity, we don’t know how to build such a device physically, 
since we want the machine operations to take the same amount of time regardless 
of the size of n; if linking is represented by using addresses as in a computer 
memory, it is necessary to put a bound on the number of nodes, since the link 
fields have a fixed size. A multitape Turing machine is therefore a more realistic 
model when n approaches infinity. Yet it seems reasonable to believe that a 
linking automaton as described above leads to a more appropriate theory of 
the complexity of algorithms than Turing machines do, even when asymptotic 
formulas for large n are considered, because the theory is more likely to be 
relevant for practical values of n. Furthermore when n gets bigger than 103° 
or so, not even a one-tape Turing machine is realistic: It could never be built. 
Relevance is more important than realism. 


Many years have passed since the author wrote most of the comments above, 
and everybody can be glad that substantial progress has indeed been made on 
the theory of linking automata (now called pointer machines). But of course 
much still remains to be done. 
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General rules for programming have been discovered. 
Most of them have been used in the 
Kansas City freight yards for a long time. 


— DERRICK LEHMER (1949) 


I must explain, to begin with, 

that all the Trees, in this system, grow head-downwards: 

the Root is at the top, and the Branches are below. 

If it be objected that the name “Tree” is a misnomer, my answer 
is that | am only following the example of all writers on Genealogy. 
A Genealogical tree always grows downwards: 

then why many not a Logical “Tree” do likewise? 


— LEWIS CARROLL, in Symbolic Logic (1896) 


You will, | am sure, agree with me... that if page 
534 finds us only in the second chapter, the length of 
the first one must have been really intolerable. 


— SHERLOCK HOLMES, in The Valley of Fear (1888) 


ANSWERS TO EXERCISES 


I am not bound to please thee with my answers. 


— SHYLOCK, in The Merchant of Venice (Act IV, Scene 1, Line 65) 


NOTES ON THE EXERCISES 
1. An average problem for a mathematically inclined reader. 


4. See W. J. LeVeque, Topics in Number Theory 2 (Reading, Mass.: Addison—Wesley, 
1956), Chapter 3; P. Ribenboim, 13 Lectures on Fermat’s Last Theorem (New York: 
Springer-Verlag, 1979); A. Wiles, Annals of Mathematics (2) 141 (1995), 443-551. 


SECTION 1.1 
1. teaatbbeccedd¢et. 


2. After the first time, the values of the variables m and n are the previous values of 
n and r, respectively; and n > r. 


3. Algorithm F (Euclid’s algorithm). Given two positive integers m and n, find 
their greatest common divisor. 


F1. 
F2. 
F3. 
F4. 


Remainder m/n.] Divide m by n and let m be the remainder. 
Is it zero?] If m = 0, the algorithm terminates with answer n. 


Remainder n/m.] Divide n by m and let n be the remainder. 


Is it zero?] If n = 0, the algorithm terminates with answer m; otherwise go 
back to step Fl. J 


4. By Algorithm E, n = 6099, 2166, 1767, 399, 171, 57. Answer: 57. 


5. Not finite nor definite nor effective, perhaps no output; in format, no letter is given 
before step numbers, no summary phrase appears, and there is no “|”. 
6. Trying Algorithm E with n = 5 and m = 1, 2, 3, 4, 5, we find that step E1 is 
executed 2, 3, 4, 3, 1 times, respectively. So the average is 2.6 = Ts. 
7. In all but a finite number of cases, n > m. And when n > m, the first iteration of 
Algorithm E merely exchanges these numbers; so Um = Tm + 1. 
466 
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8. Let A = {a,b,c}, N = 5. The algorithm will terminate with the string a2", 


j 9; Qj bj ay 

0 ab (empty) 1 2 Remove one a and one b, or go to 2. 
1 (empty) c 0 0 Add cat extreme left, go back to 0. 
2 a b 2 3 Change all a’s to b’s. 

3 c a 3 4 Change all c’s to a’s. 

4 b b 0 5 If b’s remain, repeat. 


Each iteration either decreases m or keeps m unchanged and decreases n. 


9. For example we can say C2 represents C4 if there is a function g from J; into I2, 
a function h from Q2 into Qi, and a function j from Q2 into the positive integers, 
satisfying the following conditions: 


a) If æ is in J, then A(g(x)) = z. 

b) If q is in Qo then fi(h(q)) = h(f2!(q)), where fl’! means that the function 
f2 is to be iterated j(q) times. 

c) If q is in Q2 then h(q) is in 2; if and only if q is in Qo. 


For example, let Cı be as in (2) and let C2 have Iz = {(m,n)}, 22 = {(m,n,d)}, 
Q2 = h U 22 U {(m,n, a, b,1)} U {(m, n, a,b, r,2)} U {(m, n, a, br, 3)} U {(m, n, a,b, 
r,4)} U {(m,n,a,b,5)}. Let fo((m,n)) = (m,n,m,n,1); fe((m,n,d)) = (m,n, d); 
fo((m,n,a,b,1)) = (m,n,a,b,amod b,2); fo((m,n,a,b,r,2)) = (m,n,b) if r = 0, 
otherwise (m,n, a,b,r,3); fo((m,n,a,b,7,3)) = (m,n,b,b,r,4); fo((m,n,a,b,7,4)) = 
(m,n,a,7r,5); fo((m,n, a, b,5)) = fo((m,n, a,b, 1)). 

Now let h((m,n)) = g((m,n)) = (m,n); h((m,n,d)) = (d); h((m,n, a,b, 1)) 
(a, b,0,1); h((m,n,a,b,r,2)) = (a,b,7, 2); h((m,n,a,b,r,3)) = (a,b,7,3); h((m,n, 
b,r,4)) = h(fo((m,n,a,b,7,4))); h((m,n,a,b,5)) = (a,b,b,1); j((m,n, a,b, r, 3)) 
j((m,n,a,b,r,4)) = 2, otherwise j(q) = 1. Then C2 represents C\. 


R 


Notes: It is tempting to try to define things in a more simple way — for example, 
to let g map Qı into Qe and to insist only that when 2o0,21,... is a computational 
sequence in Cı then g(xo), g(x1),... is a subsequence of the computational sequence in 
C2 that begins with g(xo). But this is inadequate; in the example above, C1 forgets 
the original value of m and n but C2 does not. 

If C2 represents C by means of functions g, h, j, and if C3 represents C2 by means 
of functions g’, h’, j’, then C3 represents C1 by means of functions g”, h”, j”, where 


g(a) =g'(g(z)), A" (a) = A(A(a)), and f"(Q)= SY), 


O<k<j(h!(q)) 


if qo = q and qk41 = y C gp). Hence the relation defined above is transitive. We 
can say C2 directly represents Cı if the function j is bounded; this relation is also 
transitive. The relation “C2 represents C1” generates an equivalence relation in which 
two computational methods apparently are equivalent if and only if they compute 
isomorphic functions of their inputs; the relation “C2 directly represents C1” generates 
a more interesting equivalence relation that perhaps matches the intuitive idea of being 
“essentially the same algorithm.” 

For an alternative approach to simulation, see R. W. Floyd and R. Beigel, The 
Language of Machines (Computer Science Press, 1994), Section 3.3. 
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SECTION 1.2.1 
1. (a) Prove P(0). (b) Prove that P(0),...,P(n) implies P(n + 1), for all n > 0. 
2. The theorem has not been proved for n = 2. In the second part of the proof, 


take n = 1; we assume there that a‘—!)~! = a~! = 1. If this condition is true (so that 
a = 1), the theorem is indeed valid. 


3. The correct answer is 1 — 1/n. The mistake occurs in the proof for n = 1, when 
the formula on the left either may be assumed to be meaningless, or it may be assumed 
to be zero (since there are n — 1 terms). 


5. If n is prime, it is trivially a product of one or more primes. Otherwise n has 
factors, so n = km for some k and m with 1 < k,m < n. Since both k and m are less 
than n, by induction they can be written as products of primes; hence n is the product 
of the primes appearing in the representations of k and m. 


6. In the notation of Fig. 4, we prove Ad implies A6. This is clear since A5 implies 
(a’ — qa)m + (b' — qb)n = (a'm + b'n) — q(am + bn) =c — qd = r. 

7. n?—(n—1)? +---—(-1)"1? =1424---4+n=n(n41)/2. 

8. (a) We must show that (n? — n +1) + (n? —n+3)+---+(n? +n — 1) equals 
n: And indeed, the sum is n(n? — n) + (1+3+4+---+(2n—-1)) = n? — n? + n°, by 
Eq. (2). But an inductive proof was requested, so another approach should be taken! 
For n = 1, the result is obvious. Let n > 1; we have (n + 1)? — (n +1) =n?—n+42n, 
so the first terms for n + 1 are 2n larger; thus the sum for n+ 1 is the sum for n plus 


an+---+2n+(n +1)? +(n+1) 1; 
— m 
n times 


this equals n? + 2n? + n? + 3n +1 = (n+ 1)°. (b) We have shown that the first 
term for (n + 1)? is two greater than the last term for n. Therefore by Eq. (2), 
18 +23 +- -- +n? = sum of consecutive odd numbers starting with unity = (number of 
terms)? = (1 +2 +- +n)’. 

10. Obvious for n = 10. If n > 10, we have 2”+! = 2.2” > (1+ 1/n)?2” and by 
induction this is greater than (1+ 1/n)?n? = (n + 1)°. 

11. (—1)"(n4+ 1)/(4(n +1)? + 1). 

12. The only nontrivial part of the extension is the calculation of the integer q in E2. 
This can be done by repeated subtraction, reducing to the problem of determining 
whether u+ v v2 is positive, negative, or zero, and the latter problem is readily solved. 

It is easy to show that whenever u + v2 = u’ +v' v2, we must have u = u’ and 
v =v’, since v2 is irrational. Now it is clear that 1 and v2 have no common divisor, 
if we define divisor in the sense that u + v v2 divides a(u+vvV2) if and only if a is an 
integer. The algorithm extended in this way computes the regular continued fraction 
of the ratio of its inputs; see Section 4.5.3. 

[Note: If we extend the concept of divisor so that u + v2 divides a(u + v2) if 
and only if a has the form u’ + v' v2 for integers u’ and v’, there is a way to extend 
Algorithm E so that it always will terminate: If in step E2 we have c = u+v v2 and d= 
u'+v'\/2, compute c/d = c(u'—v' V2) /(w/?-2u'”) = r+yv2 where z and y are rational. 
Now let q = u” + v"\/2, where u” and v” are the nearest integers to x and y; and let 
r = c—qd. Ifr = u” +v V2, it follows that ju”? —2u'””?| < ju’? —2v’?|, hence the com- 
putation will terminate. See “quadratic Euclidean domains” in number theory texts.] 
13. Add “T < 3(n—d)+k” to assertions A3, A4, A5, A6, where k takes the respective 
values 2, 3, 3, 1. Also add “d > 0” to A4. 
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15. (a) Let A = S in (iii); every nonempty well-ordered set has a least element. 

(b) Let x < y if |x| < |y| or if |x| = |y| anda < 0 < y. 

(c) No, the subset of all positive reals fails to satisfy (iii). [Note: Using the so- 
called axiom of choice, a rather complicated argument can be given to show that every 
set can be well-ordered somehow; but nobody has yet been able to define an explicit 
relation that well-orders the real numbers.] 

(d) To prove (iii) for Ta, use induction on n: Let A be a nonempty subset of Tn 
and consider Aı, the set of first components of A. Since A, is a nonempty subset of S, 
and S is well-ordered, Ai contains a smallest element x. Now consider Az, the subset 
of A in which the first component equals x; A, may be considered a subset of T,-1 
if its first component is suppressed, so by induction As contains a smallest element 
(£, £2,..., £n) that in fact is the smallest element of A. 

(e) No, although properties (i) and (ii) are valid. If S contains at least two distinct 
elements, a < b, the set (b), (a,b), (a,a,b), (a,a,a,b), (a,a,a,a,b),... has no least ele- 
ment. On the other hand T can be well-ordered if we define (x1,...,%m) < (y1,.-- Yn) 
whenever m < n, or m = n and (21,...,2n) < (y1,---,Yn) in Th. 

(f) Let S be well-ordered by <. If such an infinite sequence exists, the set A 
consisting of the members of the sequence fails to satisfy property (iii), for no element 
of the sequence can be smallest. Conversely if < is a relation satisfying (i) and (ii) but 
not (iii), let A be a nonempty subset of S that has no smallest element. Since A is not 
empty, we can find x; in A; since xı is not the smallest element of A, there is x2 in A 
for which x2 < x1; since xə is not the smallest element either, we can find x3 < z2; etc. 

(g) Let A be the set of all x for which P(x) is false. If A is not empty, it contains a 
smallest element xo. Hence P(y) is true for all y < xo. But this implies P(xo) is true, 
so xo is not in A (a contradiction). Therefore A must be empty: P(x) is always true. 


SECTION 1.2.2 
1. There is none; if r is a positive rational, r/2 is smaller. 


2. Not if infinitely many nines appear in a row; in that case the decimal expansion 
of the number is 1 + .24000000..., according to Eq. (2). 


3. —1/27, but the text hasn’t defined it. 
A. 4. 


6. The decimal expansion of a number is unique, so x = y if and only if m = n and 
di = e; for alli > 1. If x Æ y, one may compare m vs. n, dı vs. €1, d2 vs. €2, etc.; when 
the first inequality occurs, the larger quantity belongs to the larger of {z, y}. 

7. One may use induction on 2, first proving the laws for x positive, and then for x 
negative. Details are omitted here. 


8. By trying n = 0,1,2,... we find the value of n for which n™ < u < (n +1)”. 


Assuming inductively that n,di,...,d,—1 have been determined, dp is the digit such 
that 
dy dk \” dı dk ey 
ae Me ef cles eee < l EESE } i 
(r trt s) on (r 10 TOF =) 


This construction can’t make dk = 9 for all k > l, because that could happen only if 
(n+ 1/10 +---+d)/10' +1/10')” < u. 
9. ((BP/ayu/ey™ = (((pP/a)¥/¥)¥)2 = ((bP/9)™)9 = ((BP/2)2)™ = bP", hence (bP/9)"/? = 


bP"/4 This proves the second law. We prove the first law using the second: b? /apu/ v — 
(t/a Pv (pt/ a0) qu = (bi/av)petau = pp/atu/y. 
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10. If log,,2 = p/q, with p and q positive, then 21 = 10”, which is absurd since the 
right-hand side is divisible by 5 but the left-hand side isn’t. 
11. Infinitely many! No matter how many digits of x are given, we will not know 
whether 107 = 1.99999... or 2.00000..., if x’s digits agree with the digits of log,, 2. 
There is nothing mysterious or paradoxical in this; a similar situation occurs in addition, 
if we are adding .444444... to .55555.... 
12. They are the only values of di,...,dg that satisfy Eq. (7). 
13. (a) First prove by induction that if y > 0, 1+ ny < (1 +y)”. Then set y = z/n, 
and take nth roots. (b) x = b—1, n = 10". 
14. Set x = log, c in the second equation of (5), then take logarithms of both sides. 
15. Prove it, by transposing “log, y” to the other side of the equation and using (11). 
16. Inz/In 10, by (14). 
17. 5; 1; 1; 0; undefined. 
18. No, logs £ = lg x/lg8 = į lg z. 
19. Yes, since lgn < (logio n)/.301 < 14/.301 < 47. 
20. They are reciprocals. 
21. (InInz —InInb)/Inb. 
22. From the tables in Appendix A, lg a ~ 1.442695 1n z; logio x ~ .4342945 ln z. The 
relative error is ~ (1.442695 — 1.4342945)/1.442695 = 0.582%. 
23. Take the figure of area In y, and divide its height by x while multiplying its length 
by x. This deformation preserves its area and makes it congruent to the piece left when 
ln z is removed from In zy, since the height at point x + xt in the diagram for In zy is 
1/(x + xt) = (1/(1 +t))/z. 
24. Substitute 2 everywhere 10 appears. 
25. Note that z = 2~?|2?-*2| > 0, when p is the precision (the number of binary 
digits after the radix point). The quantity y + log, x stays approximately constant. 
27. Prove by induction on k that 

r” a= ee < 102% (nt01/2+: +b /2®) pt < a? (1 + go 
and take logarithms. 
28. The following solution uses the same auxiliary table as before. 

E1. [Initialize.] Set x + 1 — € — zx, y + yo, and k + 1, where 1 — e€ is the largest 
possible value of x, and yo is the nearest approximation to b'~*. (The quantity 
yb—* will remain approximately constant in the following steps.) 

E2. [Test for end.] If x = 0, stop. 

E3. [Compare.] If x < log,(2*/(2* — 1)), increase k by 1 and repeat this step. 

E4. [Reduce values.] Set x + x — log,(2*/(2* — 1)), y + y — (y shifted right k), 
and go to E2. I 

If y is set to b'*(1 + €o) in step El, the subsequent computational error arises 

when z + z + log,(1—27*) + ô; and y + y(1 — 27})(1 + €j) during the jth execution 
of step E4, for certain small errors 6; and ej. When the algorithm terminates we have 
computed y = b®7¥%; II re + €j). Further analysis depends on b and the computer 
word size. Notice that both in this case and in exercise 26, it is possible to refine the 
error estimates somewhat if the base is e, since for most values of k the table entry 
In(2*/(2* — 1)) can be given with high accuracy: It equals 2~* + $277" + 42738 Here 
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Note: Similar algorithms can be given for trigonometric functions; see J. E. 
Meggitt, IBM J. Res. and Dev. 6 (1962), 210-226; 7 (1963), 237-245. See also T. C. 
Chen, IBM J. Res. and Dev. 16 (1972), 380-388; V. S. Linsky, Vychisl. Mat. 2 (1957), 
90-119; D. E. Knuth, METAFONT: The Program (Reading, Mass.: Addison-Wesley, 
1986), §120-§147. 

29. e; 3; 4. 
30. zx. 


SECTION 1.2.3 


1. —aı; and ag +---+a; = 0. In general, sums with ‘---’ are defined so that 
(ap +--+ aq) + (Qq41 ++ + ar) = ap +--+ ar for arbitrary integers p, q, and r. 
2. ay + a2 + a3. 

3. i+4+4+4+4+ġ4;i+4+4++ +4. The rule for p(j) is violated: In the 


first case n? = 3 occurs for no n, and in the second case n? = 4 occurs for two n. [See 
Eq. (18).] 
4. (a11) + (a21 + a22) + (a31 + a32 + a33) = (a11 + a21 + a31) + (a22 + a32) + (a33). 


5. It is only necessary to use the rule a $` pa) ti = X po (ati): 
(sca) (326) =z ai i) 
R(t) S(j) R(i) S(j) R(i) \S(i) 


7. Use Eq. (3); the two limits are interchanged and the terms between ao and ae 
must be transferred from one limit to the other. 


8. Let aq+ı)i = +1, and aiq+ı) = —1, for all ¿ > 0, and all other aij zero; let 
R(i) = S(t) = “i > 0”. The left-hand side is —1, the right-hand side is +1. 
9, 10. No; the applications of rule (d) assume that n > 0. (The result is correct for 
n = —1 but the derivation isn’t.) 
11. (n+ 1)a. 
12. £(1—1/7"*"). 
13. m(n—m+1)4+5(n—m)(n—m+1); or, $(n(n + 1) — m(m — 1)). 
14. $(n(n +1) — m(m- 1))(s(s +1) — r(r -1)), ifm <n andr < s. 
15, 16. Key steps: 


5 jæi =x 5 ja =g 5 (j +1) 


0<j<n 1<jkn 0<j<n-1 
ad i , 
=g > jat —ne"t! +g > x’. 
OSj<n O<j<n-1 


17. The number of elements in S. 

18. S(j) =“1 <j <n”. R' (i,j) = “n is a multiple of i and i > j”. 

19. (an — Gm—1)[M<n]. 

20. (b—1)Ðko(n-— k)bë +n +1 = Ð gob"; this formula follows from (14) and the 
result of exercise 16. 

21. Vag 4 + Zso a = X ul RO) + Z; as [SG)] = Z; a ([RG)] + [5()]); now 
use the fact that [R(j)]+[S(/)] = [R() or S(j)|4+[ Ry) and S(j)]. In general, bracket 
notation gives us the ability to manipulate “on the line” instead of “below the line.” 


472 ANSWERS TO EXERCISES 1.2.3 


22. For (5) and (7), just change }? to] ]. We also have ] Į pi) bici = Mro bi) IRo ci) 


and 
(I=) (IT) =( [| »)( | a); 
RG) SG) RCG) or SÇ) R(j) and S(j) 


23. 0O+a2=2and1-x2=2. This makes many operations and equations simpler, such 
as rule (d) and its analog in the previous exercise. 


25. The first step and last step are OK. The second step uses i for two different 
purposes at once. The third step should probably be X ;_; n. 


26. Key steps, after transforming the problem as in Example 2: 


(Hew) E(t) = (He) (He) = (He) 


The answer is (JJ; o a)" t’. 


28. (n + 1)/2n. 

29. (a) Dockcjcicn iajak. (b) Let S, = X; o af. Solution: 353 + 55152 + $ S7. 
The general solution to this problem, as the number of indices gets larger, may be 
found in Section 1.2.9, Eq. (38). 


30. Write the left side as Vie pen QjbkTjYk, and do a similar thing on the right. 
(This identity is the special case m = 2 of exercise 46.) 


31. Set aj = uj, bj = 1, £j = vj, and yj = 1, to obtain the answer n 7", ujvj — 


i 3) (oa vj). Consequently we have (si us) (Xz vj) < n); Ujv; when 
ur < u2 < --- < un and vı < ve < +++ < Un, a result known as Chebyshev’s monotonic 
inequality. [See Soobshch. mat. obshch. Khar,kovskom Univ. 4,2 (1882), 93-98.] 


33. This can be proved by induction on n, if we rewrite the formula as 


1 ( = x5 (aj — Ln—1) £ x5 (xj — En) ) 


Ln — Ln-1 


4x1 Thice<n, egi (ti — 2k) Fy hice, ney (i — 


Each of these sums now has the form of the original sum, except on n — 1 elements, 
and the values turn out nicely by induction when 0 < r < n— 1. When r = n, consider 
the identity 


Q= 


Z JIk- (23 — zk) E y ee 


fai Dheren, ezili — te) Sa Ticken, beg (2i — r) 


where P(x;) is a polynomial of degree n — 2 in x; whose coefficients are symmetric 
functions of {x1,..., £n} that don’t depend on j. (See exercise 1.2.9-10.) We obtain 
the desired answer from the solutions for r = 0, 1, ..., n — 1. 

Notes: Dr. Matrix was anticipated in this discovery by L. Euler, who wrote to 
Christian Goldbach about it on 9 November 1762. See Euler’s Institutionum Calculi 
Integralis 2 (1769), §1169; and E. Waring, Phil. Trans. 69 (1779), 64-67. The following 
alternative method of proof, using complex variable theory, is less elementary but more 
elegant: By the residue theorem, the value of the given sum is 

1 z” dz 


2ri J\zjar (Z — £1)... (Z2 — Tn) 
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where R > |x1|,..-,|@n|. The Laurent expansion of the integrand converges uniformly 
on |z| = R; it is 


aye ae 


y= 


=2 H (ar bees + En) T] + (el + rir +e)? H. 


Integrating term by term, everything vanishes except the coefficient of z~'. This 
method gives us the general formula for an arbitrary integer r > 0: 


ji in _ 5 ) . . Z 
Ti -Ln = Tiji sss Vipin gis 
jite tin=r-n+1 Iji L Kjr-n+1 Sn 
Fyn 20 


see Eq. 1.2.9-(33). [J. J. Sylvester, Quart. J. Math. 1 (1857), 141-152. 


34. If the reader has tried earnestly to solve this problem, without getting the answer, 
perhaps its purpose has been achieved. The temptation to regard the numerators as 
polynomials in x rather than as polynomials in k is almost overwhelming. It would 
undoubtedly be easier to prove the considerably more general result 


` Iicrcn-ı (Yk = Zr) 1 


k=1 Ther<n, rk (Yr — Yr) 


which is an identity in 2n — 1 variables! 


35. If R(j) never holds, the value should be —oo. The stated analog of rule (a) is based 
on the identity a+ max(b,c) = max(a +b, a+c). Similarly if all ai, bj are nonnegative, 
we have 

SUP R(i) Qi SUP gj) bj = SUP R(;) SUP s(j) Aidj- 


Rules (b), (c) do not change; for rule (d) we get the simpler form 


sup(sup g,;) Qj, SUP 55) aj) = SUP R(j) or S(j) i- 


36. Subtract column one from columns 2,...,n. Add rows 2,...,n to row one. The 
result is a triangular determinant. 


37. Subtract column one from columns 2,...,n. Then subtract xı times row k -— 1 
from row k, for k = n, n—1, ..., 2 (in that order). Now factor xı out of the first column 
and factor x, —21 out of columns k = 2,..., n, obtaining xı(£2— z1)... (£n — x1) times 
a Vandermonde determinant of order n — 1. The process continues by induction. 
Alternative proof, using “higher” mathematics: The determinant is a polynomial 
in the variables x1,..., 2% of total degree 1+2+---+n. It vanishes if xj = 0 or if x; = 2; 
(i < j), and the coefficient of xt ae..." is +1. These facts characterize its value. In 
general, if two rows of a matrix become equal for x; = £j, their difference is usually 
divisible by x; — xj, and this observation often speeds the evaluation of determinants. 


38. Subtract column one from columns 2,...,n, and factor out 


(a1 +y)... (En + yr) (y1 — ya) ---(y1 — Yn) 
from rows and columns. Now subtract row one from rows 2,...,n and factor out 
(x1— z2)... (£1— £n) (x1 +y2) 7! yek (21 +yn)~*; we are left with the Cauchy determinant 
of order n — 1. 


39. Let I be the identity matrix (6;;), and J the matrix of all ones. Since J? = nJ, 
we have (£I + yJ)((x@+ ny)I — yJ) = x(x + ny)L. 
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40. [A. de Moivre, The Doctrine of Chances, 2nd edition (London: 1738), 197—199.] 


We have 
n 
XO bes) = Tj Il (£k = zj) Ti Il (£k = Xi) = ij. 
t=1 1<k<n 1<k<n 
ki kżi 
41. This follows immediately from the relation of an inverse matrix to its cofactors. 
It may also be interesting to give a direct proof here: We have 


n 


Trz Tj + Yk — ®)] [pyi (te + yt) 
>) Ti + we -5 Tez; ( Tj — Cr) Tee (ye — Yk) 


when x = 0. This is a polynomial of degree at most n — 1 in a. If we set x = £j + ys, 
1 < s < n, the terms are zero except when s = t, so the value of this polynomial is 


T[(-2. - 9s) / [[ (es - 2x) = [les [[(e: - 2x) 


kżi kj kżi kżŻj 


These polynomials of degree at most n — 1 agree at n distinct points x, so they agree 
also for x = 0; hence 


n 


Darn” e-a He tim Ba) Ôig: 


ki kAj 


42. n/(x + ny). 


43. 1-— Į; (1-— 1/2x). This is easily verified if any x; = 1, since the inverse of any 
matrix having a row or column all of ones must have elements whose sum is 1. If none of 
the x; equals one, sum the elements of row i by setting x = 1 in exercise 40 and obtaining 
Tevzi(te — 1)/x: [[kz:(®k — xi). After multiplying numerator and denominator by 
x; — 1, we can sum on i by applying exercise 33 with r = 0 to the n + 2 numbers 
{0,1,21,...,2n}. 

44. We find 


-Yw Testa / TT e-a 


k=1 1<k<n 
RAG 


after applying exercise 33. And 


Sas 3 (a? + (y1 +--+ + yn) +--+) 


j=l Ticken, weg ("5 =A 


= (a1 + £2 +--+ En) + (yr +yY2 +: + Yn). 


45. Let xi = i, yj = j — 1. From exercise 44, the sum of the elements of the inverse is 
(1+2+---+n)+((n—1)+(n—2)+---+0) =n?. From exercise 38, the elements 
of the inverse are 


(1 (i +n- 1)! (j+n-1)! 
(i +j- 1) = 1 — 1)! (n — i)! (n — j)! 


This quantity can be put into several forms involving binomial coefficients, for example 


DEAA EEE 
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From the latter formula we see that b;; is not only an integer, it is divisible by 7, j, n, 
i+j—1,i+n-—1,j+n-—1,n-—i+1, and n—j+1. Perhaps the prettiest formula 


for bij is 
rao) CRP YVGP): 


The solution to this problem would be extremely difficult if we had not realized 
that a Hilbert matrix is a special case of a Cauchy matrix; the more general problem is 
much easier to solve than its special case! It is frequently wise to generalize a problem 
to its “inductive closure,” i.e., to the smallest generalization such that all subproblems 
that arise in an attempted proof by mathematical induction belong to the same class. 
In this case, we see that cofactors of a Cauchy matrix are determinants of Cauchy 
matrices, but cofactors of Hilbert matrices are not determinants of Hilbert matrices. 
[For further information, see J. Todd, J. Research Nat. Bur. Stand. 65 (1961), 19-22; 
A. Cauchy, Exercices d’analyse et de physique mathématique 2 (1841), 151-159] 


46. For any integers kı, k2,...,km, let e(ki,...,km) = sign(T], <;<j<m (ki —ki)); where 
sign x = [x >0]—[x <0]. If (li,...,dm) is equal to (ki,..., km) except for the fact that 


ki and k; have been interchanged, we have e(l1,...,lm) = —e(ki,...,km). Therefore 
we have the equation det(Br,...k,) = €(ki,..-,km) det(By,...5,,), if ji < +++ < jm are 
the numbers ki,...,km rearranged into nondecreasing order. Now by definition of the 
determinant, 


det(AB) = 5 e(l, tues lm) (> aban ) sF P amabatn ) 
a k=1 k=1 
= 5 Q1ky -+ - Amkm 5 efli, ...,lm)bkih pieDigets 


1<k1,...,.km<n 1<11,--.,lm<gm 


= 5 Alki -- - Amkm det(Bki...km) 
= 5 e(ki,..-, km) Qik, ++ Amkm det(Bj,...5m) 


= 5 det(Aj,...jm) det(Bji..jm)- 


1<ji Se Ljm <N 
Finally, if ji = ji+1, det(Aj,...j) = 0. [J. de l'École Polytechnique 9 (1813), 280-354; 
10 (1815), 29-112. Binet and Cauchy presented their papers on the same day in 1812.] 
47. Let aij = (J [i] (2: +e) (pa (2: +q.)). Subtract column k—1 from column k 


and factor out pr; — qk, for k = n, n — 1, ..., j +1 (in that order), for j = 1, 
2, ..., n — 1 (in that order). This leaves [],<;2;<,(pi — qj) times det(bj;) where 
bij = [Th_j41(@i + gx). Now subtract qx+; times column k + 1 from column k for 
k = 1, ..., n— J, and for j = 1, ..., n — 1; this leaves det(ci;), where cij = rI 


essentially defines a Vandermonde matrix. We can now proceed as in exercise 37, 
operating on rows instead of columns, obtaining 


det(aij) = II (xi — ©5)(pi — qj). 
1<i<j<n 


When p; = qj = yj for 1 < j < n, the matrix in this exercise is a Cauchy matrix 
with row i multiplied by []j_,(ai + yj). Therefore this result generalizes exercise 38 
by adding n — 2 independent parameters. [Manuscripta Math. 69 (1990), 177-178.] 
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SECTION 1.2.4 
1. 1, —2, —1, 0, 5. 
2. |z]. 


3. By definition, |x] is the greatest integer less than or equal to x; therefore || is 
an integer, |x| < x, and |x| +1 >x. The latter properties, plus the fact that when m 
and n are integers we have m < n if and only if m < n — 1, lead to an easy proof of 
propositions (a) and (b). Similar arguments prove (c) and (d). Finally, (e) and (f) are 
just combinations of previous parts of this exercise. 


4. x— 1< |z| <x; s0 —x+1>—|2| > —zr; hence the result. 


5. |e +3]. The value of (—z rounded) will be the same as —(x rounded), except 


when z mod 1 = z, In the latter case, the negative value is rounded towards zero and 
the positive value is rounded away from zero. 


6. (a) is true: |vz] = n = n? < z < (n +1)? = n? < |r] < (n+ 1? = 


|vlz] | =n. Similarly, (b) is true. But (c) fails when z is, say, 1.1. 

7. |x+y] = | |z| +zmod1+|y]|+ymod1]| = |x]+ly]+|xmod1+ymod 1]. The 
inequality should be > for ceilings, and then equality holds if and only if either x or y is 
an integer or x mod 1+y mod 1 > 1; that is, if and only if (—x) mod 1+(—y) mod 1 < 1. 


8. 1, 2,5, —100. 


9. —1, 0, —2. 
10. 0.1, 0.01, —0.09. 
11. r=y. 
12. All. 
13. +1, —1. 
14. 8. 


15. Multiply both sides of Eq. (1) by z; the result is also easily verified if y = 0. 


17. As an example, consider the multiplication portion of Law A: We have a = b+ qm 
and x = y + rm, for some integers q and r; so ax = by + (br + yq + qrm)m. 


18. We have a — b = kr for some integer k, and also kr = 0 (modulo s). Hence by 
Law B, k = 0 (modulo s), so a — b = qsr for some integer q. 


20. Multiply both sides of the congruence by a’. 


21. There is at least one such representation, by the previously proved exercise. If there 
are two representations, n = pı ... Pk = qı - -- qm, we have qi...Gm = 0 (modulo pı); 
so if none of the q’s equals pı we could cancel them all by Law B and obtain 1 = 0 
(modulo pi). The latter is impossible since pı is not equal to 1. So some q; equals pi, 
and n/p, = p2...Pk = Q---Gj-19j+1---m-. Either n is prime, when the result is 
clearly true, or by induction the two factorizations of n/p are the same. 


22. Let m = ax, where a > 1 and z > 0. Then ax = 0 but x Æ 0 (modulo m). 
24. Law A is always valid for addition and subtraction; Law C is always valid. 
26. If b is not a multiple of p, then b? — 1 is, so one of the factors must be. 


27. A number is relatively prime to p° if and only if it is not a multiple of p. So we 


count those that are not multiples of p and get y(p*) = p° — p®~*. 
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28. If a and bare relatively prime to m, so is ab mod m, since any prime dividing the 
latter and m must divide a or b also. Now simply let 21,...,2,(m) be the numbers 
relatively prime to m, and observe that azı modm,..., a2 ,(m) mod m are the same 
numbers in some order, etc. 


29. We prove (b): If r L s and if k? divides rs, then p° divides rs for some prime p, 
so p divides r (say) and cannot divide s; so p? divides r. We see that f(rs) =0 if and 
only if f(r) =0 or f(s) =0. 


30. Suppose r L s. One idea is to prove that the y(rs) numbers relatively prime to rs 


are precisely the y(r)y(s) distinct numbers (sx; + ryj) mod (rs) where 11,...,%,(r) 
and yi,.--,Yy(s) are the corresponding values for r and s. 
Since y is multiplicative, (10°) = y(2°)y(5°) = (2° — 2°)(5° — 5°) = 400000. 


And in general when n = pî! ... pê", we have y(n) = (pt — p§!7')... (pe — p&r71) = 


21] ,\n,pprime(1 — 1/p). (Another proof appears in exercise 1.3.3-27.) 


31. Use the fact that the divisors of rs may be uniquely written in the form cd where 
c divides r and d divides s. Similarly, if f(n) > 0, one can show that the function 
MaXa\n f(d) is multiplicative (see exercise 1.2.3-35). 


33. Either n +m or n— m + 1 is even, so one of the quantities inside the brackets is 
an integer; so equality holds in exercise 7, and we obtain (a) n; (b) n +1. 


34. b must be an integer > 2. (Set x = b.) The sufficiency is proved as in exercise 6. 
The same condition is necessary and sufficient for [log, x] = [log,[z]]. 

Note: R. J. McEliece has pointed out the following generalization: Let f be a 
continuous, strictly increasing function defined on an interval A, and assume that both 
|x| and [x] are in A whenever z is in A. Then the relation | f(x)| = | f(|z])| holds 
for all z in A if and only if the relation [f(x)] = [f(fz])] holds for all x in A, 
if and only if the following condition is satisfied for all x in A: “f(ax) is an integer 
implies x is an integer.” The condition is obviously necessary, for if f(x) is an integer 
and it equals | f(|x|)| or [f([{z])] then x must equal |x| or [x]. Conversely if, say, 
|f(lx])] < Lf(x)| then by continuity there is some y with |x| < y < x for which f(y) 
is an integer; but y cannot be an integer. 


35, ZM _,_ zm 1 n-1 é |e] +m n-1 < ete < em 


n n n n n n n 


apply exercise 3. Use of exercise 4 gives a similar result for the ceiling function. Both 
identities follow as a special case of McEliece’s theorem in exercise 34. 


36. Assume first that n = 2t. Then 


hence 


n 


Bilas) a ea 


k=1 k=1 k=1 


by exercise 33. And if n = 2t + 1, we have t? + |n/2] = t? +t = n?/4—1/4. For the 
second sum we get, similarly, [n(n + 2)/4]. 
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=] 
37. 5 met raeo +a. Let {y} denote y mod 1; we must subtract 


2 
0<k<n 
mk +z 
s= 2 {mee 


O<k<n 


This quantity S consists of d copies of the same sum, since if t = n/d we have 
mk +a mk+t) +2 
n 7 n ` 


H nal 


O<k<t O<k<t 


Let u = m/d; then 


and since t L u this sum may be rearranged to equal 


x mod d x mod d ne D i x mod d _t-1 
n T n T t T T n T t . 


Finally, since (x mod d)/n < 1/t, the braces in this sum may be removed and we have 


s = a (“ened = 


n 2 


An application of exercise 4 yields the similar identity 


> ea) ee D RL ajail 


n 2 
0<k<n 


This formula would become symmetric in m and n if it were extended over the range 
0< k <n. (The symmetry can be explained by drawing the graph of the summand as 
a function of k, then reflecting about the line y = zx.) 


38. Both sides increase by [y] when x increases by 1, so we can assume that 0 < x < 1. 
Then both sides are zero when x = 0, and both sides increase by 1 when x increases 
past the values 1 — k/y for y > k > 0. [Crelle 136 (1909), 42; the case y = n is due to 
C. Hermite, Acta Math. 5 (1884), 315.] 


39. Proof of part (f): Consider the more general identity | [o<p<n2sin T(x + k/n) 
2sin rng, which can be demonstrated as follows: Since 2sin@ = (e? — e~"*)/i 
(1 — e7 7") e’*-*"/2. the identity is a consequence of the two formulas 


Il (1 = errs) = p= en atne and II et (e— (1/2) +(k/n)) = et (ne—1/2) 
O<k<n O0<k<n 


The latter is true since the function x — Z is replicative; and the former is true because 
we may set z = 1 in the factorization of the polynomial z” — a” = (z — a)(z — wa)... 
m—1q), where w = e7?™/”, 


(z= w 
40. (Note by N. G. de Bruijn.) If f is replicative, f (nx + 1) — f(nx) = f(x + 1) — f(x) 
for all n > 0. Hence if f is continuous, f(x +1) — f(x) = c for all x, and g(x) = 
f(x) — elx] is replicative and periodic. Now 


1 34i 1 L al 
f es g(x) dx = =f en" g(y) dy; 
0 0 


n 
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expanding in Fourier series shows that g(x) = (x — $)a for 0 < x < 1. It follows 
that f(x) = (« — $)a. In general, this argument shows that any replicative locally 
Riemann-integrable function has the form («—4)a+bmax(|x|,0)+cmin(|z],0) almost 
everywhere. For further results see L. J. Mordell, J. London Math. Soc. 33 (1958), 371- 


375; M. F. Yoder, Æquationes Mathematicze 13 (1975), 251-261. 
41. We want an = k when $k(k—1) <n < 5k(k+1). Since n is an integer, this is 
equivalent to 
kK(k—-1 1 k(k+1) , 1 

i a ee. 
ie., k— $ < V2n<k+ 5. Hence an = |V2n+ 4 |, the nearest integer to /2n. Other 
correct answers are [V/2n — 4], [(/8n +1 -1)/2], |(vV8n =7 + 1)/2], etc. 
42. (a) See exercise 1.2.7-10. (b) The given sum is n|log, n] — S, where 

s= DY k= J} H-1 = peH iG — [log, nl. 


1l<k<n 1<t<log, n 
k+1 is a power of b 


43. [vn] (n= §(2LVn] +5)(Lvn] — 1). 


44. The sum is n+ 1 when n is negative. 


1 
45. |mj/n| = r if and only if [=] <j< and we find that the given 


sum is therefore 
ae le 


Osram 


The stated result follows by rearranging the latter sum, grouping the terms with a 
particular value of [rn/m]. The second formula is immediate by the substitution 


z+1 
Hasi): 
46. Yocjcan F(MI/n]) = Locrcam|tn/m|(F(r — 1) — f(r)) + fan] f(fam] — 1). 
47. (a) The numbers 2, 4, ..., p— 1 are the even residues (modulo p); since 2kq = 


p|2kq/p| + (2kq) mod p, the number (—1)!?*4/?! ((2kq) mod p) will be an even residue 
or an even residue minus p, and each even residue clearly occurs just once. Hence 
(—1)7g®-Y/22.4...(p—1) =2-4...(p—1). (b) Let q = 2. If p= 4n + 1, o = n; if 
p= 4n +3, o = n+ 1. Hence (5) = (1,—1,—1,1) according as pmod 8 = (1,3,5,7), 
respectively. (c) For k < p/4, we have 


|(p—1—2k)q/p| = q- [(2k+1)q/p] = q—-1—-|(2k+1)q/p] = [(2k+1)q/p] (modulo 2). 


Hence we may replace the last terms |(p — 1)¢/p], |(p — 3)q/p],... by |a/p], |3a/p], 
ete. (d) Mocrepalka/Pl + Mo<r<qs2/™P/al = [p/2](14/2] — 1) = as + 1)(q— 1)/4. 
Also Vo<r<q/2/7P/41 = X o<r<q/21"P/4] + (q — 1)/2. The idea of this proof goes back 
to G. Eisenstein, Crelle 28 (1844), 246-248; Eisenstein also gave several other proofs 
of this and other reciprocity laws in the same volume. 


48. (a) This is clearly not always true when n < 0; when n > 0 it is easy to verify. 


(b) [(n + 2— |n/25])/3] = [(n — |n/25])/3] = [(n + [—n/25])/3] = [[24n/25]/3] = 
[8n/25] = |(8n + 24)/25|. The penultimate equality is justified by exercise 35. 
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49. Since f(0) = f(f(0)) = F(f(0) +0) = F(0) 
integers n. If f(4) =k < 0, we have k = f(r m 


f(0), we have f(n) = n for all 


3 oe = ELS (fG) k)) = 
f(0) = 0. And if f(;4) = 0 we have f(+) = f(4 Oe 2) = f(s) = 0; 
furthermore 1 < m < n implies f(™) = Ae (*)) = f(4) = 0, for a = [n/m], 
by induction on m. Thus f(5) < 0 implies a y= |z] Ae a rational x. On the 
other hand, if f(5) > 0 the function g(x) = —f(—z) satisfies (i) and (ii) and has 
9($) = 1 — f(4) < 0; hence f(x) = —g(—x) = —|-2] = [x] for all rational z. 


[P. Eisele and K. P. Hadeler, AMM 97 (1990), 475-477.] 

It does not follow, however, that f(x) = |x] or [x] for all real values of x. If, for 
example, h(x) is any Sine with h(1) = 1 and h(x + y) = h(x) + A(y) for all real x 
and y, then the function f(x) = | h(x)| satisfies (i) and (ii); but h(x) may be unbounded 
and highly erratic when 0 i x< 1 [G. Hamel, Math. Annalen 60 (1905), 459-462]. 


SECTION 1.2.5 


1. 52!. For the curious, this number is 806 58175 17094 38785 71660 63685 64037 
66975 28950 54408 83277 82400 00000 00000. (!) 


2. Pnk = Pn(k—1)(n — k + 1). After the first n — 1 objects have been placed, there is 
only one possibility for the last object. 


3. 53124, 35124, 31524, 31254, 31245; 42351, 41352, 41253, 31254, 31245. 


4. There are 2568 digits. The leading digit is 4 (since log,, 4 = 2 log,, 2 œ .602). The 
least significant digit is zero, and in fact by Eq. (8) the low order 249 digits are all zero. 
The exact value of 1000! was calculated by H. S. Uhler using a desk calculator and much 
patience over a period of several years, and appears in Scripta Mathematica 21 (1955), 
266-267. It begins with 402 38726 00770 .... (The last step in the calculation, to 
multiply the two numbers 750! and i aon k, was performed on UNIVAC I by John W. 
Wrench, Jr., “in the extraordinary time of 21/2 minutes.” Nowadays, of course, a 
desktop imachihe easily produces 1000! in a fraction of a second, and we can confirm 
that Uhler’s value was 100% correct.) 


5. (39902)(97/96) ~ 416 + 39902 = 40318. 

6. 218.38. 54.7?.11-13-17-19. 

8. It is limmo M? m!/((n + m)!/n!) = n! limmo M”/((m + 1)... (m +n)) =n, 
since m/(m + k) > 1. 


9. yr and —2,/7. (Exercise 10 used.) 


10. Yes, except when <z is zero or a negative integer. For we have 


i m” m! m 
a a Tn (3): 
11,12. p= (agp*'+---+a1)+(anp 7 +- +42) +--+ tak 
= ak(pP" +--+ +p +1) +--+ +41 = (alp — 1) +--+ alp — 1))/(p— 1) 
= (n— ak —-+-+—a1—a0)/(p—1). 


13. For each n, 1 < n < p, determine n’ as in exercise 1.2.4-19. There is exactly 
one such n’, by Law 1.2.4B; and (n’)/ = n. Therefore we can pair off the numbers in 
groups of two, provided that n’ # n. If n! = n, we have n? = 1 (modulo p); hence, as 
in exercise 1.2.4-26, n = 1 orn = p— 1. So (p—1)!=1-1...1-(-—1), since 1 and p— 1 
are the only unpaired elements. 


1.2.5 ANSWERS TO EXERCISES 481 


14. Among the numbers {1,2,...,n} that are not multiples of p, there are |n/p| 
complete sets of p — 1 consecutive elements, each with a product congruent to —1 
(modulo p) by Wilson’s theorem. There are also ao left over, which are congruent 
to ao! (modulo p); so the contribution from the factors that are not multiples of p is 
(—1)'"/?lao!. The contribution from the factors that are multiples of p is the same as 
the contribution in |n/p|!; this argument can therefore be repeated to get the desired 
formula. 


15. (n!)3. There are n! terms. Each term has one entry from each row and each 
column, so it has the value (n!)?. 


16. The terms do not approach zero, since the coefficients approach 1/e. 


17. Express the gamma functions as limits by Eq. (15). 


n n rr) 2 
18. Ia TAT artsy 


[Wallis’s own heuristic “proof” can be found in D. J. Struik’s Source Book in Mathe- 
matics (Harvard University Press, 1969), 244-253.] 


19. Change of variable t = mt, integration by parts, and induction. 


20. [For completeness, we prove the stated inequality. Start with the easily verified 
inequality 1+ < e”; set x =+t/n and raise to the nth power to get (lL4t/n)” < e*t. 
Hence e™ > (1—t/n)” = e™(1—t/n)”e > e*(1—-t/n)"(14+t/n)” = e*(1—-#?/n?)” > 
e ‘(1 — t?/n) by exercise 1.2.1-9.] 

Now the given integral minus I(x) is 


f etr! a+ f Ca = (1 = ~) ) t! dt. 
m 0 m 


As m — oo, the first of these integrals approaches zero, since t”~' < e 
and the second is less than 


1s” 1 f” 
= f tte tdt< — f tte * dt +0. 
m 0 m 0 


21. If c(n, j, kı, k2,...) denotes the appropriate coefficient, we find 


t/2 for large t; 


e(n+1,j,ki,...) = e(n,j—1,k1—-1,ke,...)+ (ki +1) e(n, j, ki +1, k2—1, k3,...) 
+(ke+1)c(n,j,ki,ko+1,k3—-1,ka,...)+---, 


by differentiation. The equations kı + k2+---= j and ki +2k2+--- = n are preserved 
in this induction relationship. We can easily factor n!/(k1!(1!)*1 k2! (2!)"2....) out of 
each term appearing on the right-hand side of the equation for c(n +1,j,k1,...), and 
we are left with kı + 2k2 + 3k3 + --- = n + 1. (In the proof it is convenient to assume 
that there are infinitely many k’s, although clearly kn41 = kn+2 = <+- = 0.) 


The solution just given makes use of standard techniques, but it doesn’t give 
a satisfactory explanation of why the formula has this form, nor how it could have 
been discovered in the first place. Let us examine this question using a combinatorial 
argument suggested by H. S. Wall [Bull. Amer. Math. Soc. 44 (1938), 395-398]. Write 
for convenience wj = Diw, up = Dku. Then D.(w;) = wj41u1 and Dz(up) = Uk+1- 


482 ANSWERS TO EXERCISES 1.2.5 


By these two rules and the rule for derivative of a product we find 


1 
D w = wiu 
2 
Dw = (wuu, + wiu2) 


D2w a ((wururu + wzuzuı + weuru2) + (w2u1u2 + wius)), etc. 
Analogously we may set up a corresponding tableau of set partitions thus: 
D' = {1} 
D? = ({2H1} + {2,1}) 
D? = (({3H2H 1 + {3,2H1 + {2H3, 1}) + ({3H{2, 1} + (3,2, 1})), ete. 


Formally, if aja2...a; is a partition of the set {1,2,...,n— 1}, define 


Dajaz...aj = {n}aia2...aj + (a1 U{n})az...a; 
+ ai(a2 U {n})...a; +--+ + aiaz... (a; U {n}). 

This rule is an exact parallel of the rule 
Dy (Wj Ury Urs aid Ur; ) = W5j41U1 Ur, Ura +++ Ur; + WjUrı+1Uro +++ Ur; 


J 


+ WjUrı Urgt1 +--+ Ur; 


j +e + WjUr Urg bs -Urj+1, 


if we let the term wjtr, Urs -- Ur; correspond to a partition a1a2...a; with rg elements 
in a, 1 < t < j. So there is a natural mapping from D” onto Dfw, and furthermore 
it is easy to see that D” includes each partition of the set {1,2,...,n} exactly once. 
(See exercise 1.2.6-64.) 

From these observations we find that if we collect like terms in Dw, we obtain a 
sum of terms c(k1, ka, . .. wut uke ..., where 7 = ki +k2+--- and n = kı +2k2 +-->, 
and where c(ki,k2,...) is the number of partitions of {1,2,...,n} into j subsets such 
that there are ky subsets having t elements. 

It remains to count these partitions. Consider an array of kų boxes of capacity t: 


k3 
ko raa 


————Sa a, 
kı a 
—_ 


The number of ways to put n different elements into these boxes is the multinomial 
coefficient 


( n )= n! 
1,1,...,1,2,2,...,2,3,3,...,3,4,.../ Fr 2]a3lhs 


To get c(ki, k2,k3,...) we should divide this by ki! kə! k3!..., since the boxes in each 
group of k; are indistinguishable from each other; they may be permuted in k! ways 
without affecting the set partition. 

Arbogast’s original proof [Du Calcul des Dérivations (Strasbourg: 1800), §52] was 
based on the fact that DEu/k! is the coefficient of z* in u(x + z) and D2w/¥j! is the 
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coefficient of y? in w(u + y), hence the coefficient of z” in w(u(a + z)) is 


Dtw a5 > j! Diu\" (D2u\" (Du 
n a j ki! kəl... kna! ( T! ep Val yp 
j=0 ky tket+--+kn=j 


ky +2kə+ e +nkn=n 
k1,k,...,kn20 


His formula was forgotten for many years, then rediscovered independently by F. Faa 
di Bruno [Quarterly J. Math. 1 (1857), 359-360], who observed that it can also be 
expressed as a determinant 


Cga CT) C 


2 n—2 n—1 
-1 (u (Puz o Cee (P3) Un- 
D? = det 0 -1 At yi (Riun (R73) Un—2 
0 0 0 axe -1 (Jur 


where u; = (Diu) Du; both sides of this equation are differential operators to be applied 
to w. For a generalization of Arbogast’s formula to functions of several variables, and 
a list of references to other related work, see the paper by I. J. Good, Annals of 
Mathematical Statistics 32 (1961), 540-541. 

22. The hypothesis that limn—o.0(n+z)!/(n!n*) = 1 is valid for integers x; for example, 
if x is positive, the quantity is (1+1/n)(1+2/n)...(1+a/n), which certainly approaches 
unity. If we also assume that xz! = x(x — 1)!, the hypothesis leads us to conclude 
immediately that 


l 
i= i (n+x)! =al ii (x+1)...(£+n) 


n>œ nin® n— oo n! n? 


? 


which is equivalent to the definition given in the text. 
23. 2(—2)!I'(z) = limmoo [[™, (1 — z/n)7"(1 + z/n)“ by (13) and (15). 
24.0 = (Goes DR < Jie; ae = Tae ese a < 
n—1 —1 
k=1 © 


25. rtr = g™ (g — m)”; et" = g™(æ +m)”. These laws hold also when m and n 
are nonintegers, by (21). 


SECTION 1.2.6 
1. n, since each combination leaves out one item. 
2. 1. There’s exactly one way to choose nothing from the empty set. 
3. Cas The actual number is 635013559600. 
4. 2*.57 . 7? 17-23-4143 - 47, 
5 
6 


. (10 + 1)* = 10000 + 4(1000) + 6(100) + 4(10) + 1. 
. r=-—3: 1-3 6 —10 15 —21 28 —36 ... 
r=—2: 1-2 3 4 5 -6 7 -8... 
r=-1: 1-1 1 1 1 -1 1 -1... 
7. |n/2]; or, alternatively, [n/2]. It is clear from (3) that for smaller values the 
binomial coefficient is strictly increasing, and afterwards it decreases to zero. 
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8. The nonzero entries in each row are the same from left to right as from right to left. 
9. One if n is positive or zero; zero if n is negative. 
10. (a), (b) and (f) follow immediately from (e); (c) and (d) follow from (a), (b), and 
Eq. (9). Thus it suffices to prove (e). Consider (ọ) as a fraction, given by Eq. (3) with 
factors in numerator and denominator. The first k mod p factors have no p’s in the 
denominator, and in the numerator and denominator these terms are clearly congruent 


to the corresponding terms of 
Y mod r) 
k mod p/’ 


which differ by multiples of p. (When dealing with non-multiples of p we may work 
modulo p in both numerator and denominator, since if a = c and b = d and a/b, c/d 
are integers, then a/b = c/d.) There remain k — k mod p factors, which fall into |k/p] 
groups of p consecutive values each. Each group contains exactly one multiple of p; 
the other p — 1 factors in a group are congruent (modulo p) to (p — 1)! so they cancel 
in numerator and denominator. It remains to investigate the |k/p| multiples of p in 
numerator and denominator; we divide each of them by p and are left with the binomial 


coefficient |(n — k mod p) /p| 
( n oe P)/P J: 


( [n/p] ) 
[k/p] 
as desired; and if k mod p > n mod p, the other factor aa a ) is zero, so the formula 


holds in general. [American J. Math. 1 (1878), 229-230; see also L. E. Dickson, Quart. 
J. Math. 33 (1902), 383-384; N. J. Fine, AMM 54 (1947), 589-592.] 


If k mod p < n mod p, this equals 


11. If a = arp” +- +ao, b = brp” +---+ bo, and a + b = crp” +---+ co, the value 
of n (according to exercise 1.2.5-12 and Eq. (5)) is 
(ao +--+ + ar +bo +: + br — co — +++ — cr)/(p— 1). 


A carry decreases c; by p and increases cj+1 by 1, giving a net change of +1 in this 
formula. [Similar results hold for q-nomial and Fibonomial coefficients; see Knuth and 
Wilf, Crelle 396 (1989), 212-219.] 


12. By either of the two previous exercises, n must be one less than a power of 2. More 
generally, w) is never divisible by the prime p, 0 < k < n, if and only if n = ap™ — 1, 
1<a<p,m>0. 
n+1 n+1 n+1 n+1 
14. 24 14 
E ee 
n? ni n® n n(n +1)(n+ 4)(3n? + 3n—-1) 


3 30 15 


15. Induction and (9). 


17. We may assume that r and s are positive integers. Also 


for all x, so the coefficients of x” must be identical. 
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21. The left-hand side is a polynomial of degree < n; the right-hand side is a polyno- 
mial of degreem-+n-+1. The polynomials agree at n+ 1 points, but that isn’t enough 
to prove them equal. [In fact, the correct formula in general is 


ee a ae 


=0 


when m, n, and r are nonnegative integers.] 


22. Assume that n > 0. The kth term is r/(r — tk) times 


T I c-t-) [| mn-1-r+tk-j) 


O<j<k O0<j<n—-k 
=I k-1 n ; ; 
<a [[ Cr+tk+3) [[ (Cr+tk+3) 


O<j<k k<j<n 
and the two products give a polynomial of degree n — 1 in k after division by r — tk. 
So the sum over k is zero by Eq. (34). 


24. The proof is by induction on n. If n < 0 the identity is obvious. If n > 0, we 
prove it holds for (r,n —r+nt+m,t,n), by induction on the integer m > 0, using the 
previous two exercises and the validity for n— 1. This establishes the identity (r, s, t, n) 
for infinitely many s, and it holds for all s since both sides are polynomials in s. 


25. Using the ratio test and straightforward estimates for large values of k we can 
prove convergence. When w is sufficiently small, we have 


= Do) a Lo REG") 


k,j J 
=e CP Cay = T 


Now let x = 1/(1+w), z = —w/(1+w)'*". This proof is due to H. W. Gould [AMM 63 
(1956), 84-91]. See also the more general formulas in exercises 2.3.4.4-33 and 4.7-22. 


26. We could start with identity (35) in the form 


mo") =" 


J 


and proceed as in exercise 25. Another way is to differentiate the formula of that 
exercise with respect to z; we get 


ed") (£t — 2" )re" 
ye z= GLa — ta 


hence we can obtain the value of 


Di — k) Ax(r, t)z". 


k 


27. For Eq. (26), multiply the series for x”*1/((t + 1)a — t) by the series for zê, and 
get a series for 2”**t1/((t + 1)a — t) in which coefficients of z may be equated to the 
coefficients arising from the series for 2tS)*1/((t + 1)a — t). 
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28. Denoting the left-hand side by f(r, s,t,n), we find 


($S) sep se 1, s—t, t, n—1)= f(r,s,t,n) 


by considering the identity 


r+tk s—tk r r+tk s—tk tk 
= t,n). 
a k ede aA k Wor a Pata) 


30. Apply (7), (6), and (19) to get oan 
a (ae 
Mk k J 2k+1 ` 

k>0 


Now we can apply Eq. (26) with (r, s,t,n) = (1,m — 2n — 1, —2,n — m), obtaining 


n-m / —m n—1 
aa 
n-m n-m 
This result is the same as our previous formula, when n is positive, but when n = 0 the 


answer we have obtained is correct while (ea is not. Our derivation has a further 


bonus, since the answer (R) is valid for n > 0 and all integers m. 


31. [This sum was first obtained in closed form by J. F. Pfaff, Nova Acta Acad. Scient. 
Petr. 11 (1797), 38-57.] We have 


ee a | 
es es 
ag a 


m+n—-j 
nj 


Changing ( ) to (t771) and applying (20) again, we get 


Tran lcm cic, 
32. Replace x by —z in (44). 


33, 34. [Mém. Acad. Roy. Sci. (Paris, 1772), part 1, 492; C. Kramp, Elémens d’Arith- 
métique Universelle (Cologne: 1808), 359; Giornale di Mat. Battaglini 33 (1895), 179- 
182.] Since z” = n! Ce), the equation may be transformed into 


ae a? Gear ole ete ee k)(1 2) 2 


which is a case of (26). Similarly, (x + y)? = >>, (Z)z(x — kz — 1)#== (y + kz)”, an 
equivalent formula of Rothe [Formulze de Serierum Reversione (Leipzig: 1793), 18]. 


35. For example, we prove the first formula: 


yeu [kl J Ba) zë = ng? + rr? = ott}, 


k 
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36. By (13), assuming that n is a nonnegative integer, we get 2” and 6,0, respectively. 
37. When n > 0, 2”~*. (The odd and even terms cancel, so each equals half the total 
sum.) 

38. Let w = e?7/™, Then 


D ajut SD (an, 


O<j<m t O<j<m 


Now l 
5 w'? = m[r=0 (modulo m)] 


(it is the sum of a geometric progression), so the right-hand sum is m 5 o 


t mod m=k 
The original sum on the left is 


Y ap n y (2c0s Z) u0, 
m 


0<j<m O0<j<m 


Since the quantity is known to be real, we may take the real part and obtain the stated 
formula. [See Crelle 11 (1834), 353-355.] 

The cases m = 3 and m = 5 have special properties discussed in CMath, exercises 
5.75 and 6.57. 
39. n!; dno — 6n1. (The row sums in the second triangle are not so simple; we will find 
(exercise 64) that $, {7} is the number of ways to partition a set of n elements into 
disjoint sets, which is the number of equivalence relations on {1,2,...,n}.) 


40. Proof of (c): By parts, 


t?(1—#)¥ |! 


Bia +1, y)= F 


$ 
' zf (1 — t)” dt. 
0 


o y 


Now use (b). 

41. m”B(z, m+ 1) > I(x) as m > o, regardless of whether m runs through integer 
values or not (by monotonicity). Hence, (m + y)”B(z, m + y + 1) > T(x), and 
(m/(m-+y))” > 1. 

42. 1/((r + 1)B(k + 1, r — k + 1)), if this is defined according to exercise 41 (b). In 
general when z and w are arbitrary complex numbers we define 


! 
the value is infinite when z is a negative integer and w is not an integer. 

With this definition, the symmetry condition (6) holds for all complex n and k, 
except when n is a negative integer and k is an integer; Eqs. (7), (9), and (20) are 
never false, although they may occasionally take indeterminate forms such as 0- œo or 
oo + oo. Equation (17) becomes 


(7) = seas z ze Zz ") 
w sin Tz w ` 


We can even extend the binomial theorem (13) and Vandermonde’s convolution (21), 


obtaining 57, (air z%t¥ = (1+ 2z)" and >, (air (ai = CE: these formulas hold 


for all complex r, s, z, a, and 8 whenever the series converge, provided that complex 
powers are suitably defined. [See L. Ramshaw, Inf. Proc. Letters 6 (1977), 223-226.] 
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43. ih dt 21 _ t)\/?) — 2f du/(1 _ u?) t? = 2 arcsin ul = TT. 


1 [r 1 (l—k/r)* 1 
45. For large r, ETN Tk & (1 k/ry > TFD 


z y 
46. ,/2 (- =) (1 2) (1 =) , and oa x A"//an. 
2r\n y xz y n 
47. Each quantity is ôko when k < 0, and is multiplied by (r — k)(r — 4 — k)/(k + 1)? 
when k is replaced by k +1. When r = — 4 this implies (x3) = (-1/ wee? 


48. This can be proved by induction, using the fact that 
n k ~ /n\ (—1)*k o 1)" 
0= -p= 
D ) (i) k+2 t2 k) k+z 
when n > 0. Alternatively, we have 


B(s, n +1) = f POG t)" dt = Bie (=1)* ‘A (t81 ap, 


0 


(In fact, the stated sum equals B(x, n + 1) for noninteger n also, when the series 
converges. ) 


49. (") = Eh) PaA (—1)™**, integer m. (See exercise 17.) 


50. The kth summand is (%)(—1)”~ "(a — kz)”~*x. Apply Eq. (34). 
51. The right-hand side is 


El á „ee = Eà" etet kz)” 7*7 
=D(F)e@tw OG i p) -re key 


= D)e +y)0" = (£ +y)”. 


The same device may be used to prove Torelli’s sum (exercise 34). 
Another neat proof of Abel’s formula comes from the fact that it is readily trans- 
formed into the more symmetric identity derived in exercise 2.3.4.4-29: 


E (7) eetk yly + (n= Ra = (wt (wt yt ney", 


Abel’s theorem has been generalized even further by A. Hurwitz [Acta Mathema- 
tica 26 (1902), 199-203] as follows: 


So a(x + e121 tos + enn) T Tiy e121 — +++ enn) P= (wt y)” 
where the sum is over all 2” choices of €1,...,€, = 0 or 1 independently. This is an 
identity in x,y, 21,...,2n, and Abel’s formula is the special case z1 = z2 =-+: = Zn. 


Hurwitz’s formula follows from the result in exercise 2.3.4.4—30. 
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52. ,s9(kK +1)? = 27/6. [M. L. J. Hautus observes that the sum is absolutely 
convergent for all complex x, y, z, n whenever z Æ 0, since the terms for large k 
are always of order 1/k”. This convergence is uniform in bounded regions, so we may 
differentiate the series term by term. If f(x, y,7) is the value of the sum when z = 1, we 
find (0/dy) f(x, y,n) = nf (x, y, n— 1) and (0/02) f(x, y, n) =nf(x—1,y+1,n-—1). 
These formulas are consistent with f(x,y, n) = (a+y)”; but actually the latter equality 
seems to hold rarely, if ever, unless the sum is finite. Furthermore the derivative with 
respect to z is almost always nonzero.| 
53. For (b), set r = 4 and s = —3 in the result of (a). 
54. Insert minus signs in a checkerboard pattern as shown. 
1 —0 0 -0 
—1 1 —0 0 
1 -2 1 —0 
—1 3 —3 1 


This is equivalent to multiplying a;; by (—1)‘*’. The result is the desired inverse, by 

Eq. (33). 

55. Insert minus signs in one triangle, as in the previous exercise, to get the inverse of 

the other. (Eq. (47).) 

56. 210 310 320 321 410 420 421 430 431 432 510 520 521 530 531 532 540 541 542 

543 610. With a fixed, b and c run through the combinations of a things two at a time; 

with a and b fixed, c runs through the combinations of b things one at a time. 
Similarly, we could express all numbers in the form n = (4) } (3) } (5) } (2) with 

a > b > c> d> 0; the sequence begins 3210 4210 4310 4320 4321 5210 5310 5320 . 

We can find the combinatorial representation by a “greedy” method, first choosing the 

largest possible a, then the largest possible b for n — Gy etc. [Section 7.2.1.3 discusses 

further properties of this representation. ] 


58. [Systematisches Lehrbuch der Arithmetik (Leipzig: 1811), xxix.] Use induction 


and 
k q k q k 1 q k q k 1 q i 


Therefore [F. Schweins, Analysis (Heidelberg: 1820), §151] the q-generalization of (21) 


is 
r s (r—k)(n—=k) _ Gat s ) are) 
po aT ae ae an= k)i n Ji 


k k 


And the identity 1 — q* = —q (1 — q7™*) makes it easy to generalize (17) to 


Tya gask ao kr—k(k—1)/2 
(1), = 6 y*( k hl 


The q-nomial coefficients arise in many diverse applications; see, for example, Section 
5.1.2, and the author’s note in J. Combinatorial Theory A10 (1971), 178-180. 

Useful facts: When n is a nonnegative integer, (dy is a polynomial of degree 
k(n — k) in q with nonnegative integer coefficients, and it satisfies the reflective laws 


R rr) ae 


If |q| < 1 and |z| < 1, the gnomial theorem holds when n is an arbitrary real number, 
if we replace the left-hand side by [],5)((1+¢*°z)/(1+q"**z)). Properties of power 


490 ANSWERS TO EXERCISES 1.2.6 


series make it necessary to verify this only when n is a positive integer, because we can 
set q” = y; the identity has then been verified for infinitely many values of y. Now we 
can negate the upper index in the q-nomial theorem, obtaining 


(1 — qattig) os ’) k(k—1)/2 r+l ak k+r k 

a-ho _ MDE gay a (PEN) a 
lI (1 — q*x) 2 k q >, k J4 
For further information, see G. Gasper and M. Rahman, Basic Hypergeometric Series 
(Cambridge Univ. Press, 1990). The q-nomial coefficients were introduced by Gauss in 
Commentationes societatis regiæ scientiarum Gottingensis recentiores 1 (1808), 147— 
186; see also Cauchy [Comptes Rendus Acad. Sci. 17 (Paris, 1843), 523-531], Jacobi 
[Crelle 32 (1846), 197-204], Heine [Crelle 34 (1847), 285-328], and Section 7.2.1.4. 


59. (n+ DE a (ata): 


60. (" y g 7 A: This formula can be remembered easily, since it is 


n(n +1)... (n+k-— 1) 
k(k—1)...1 i 


like Eq. (2) except that the numbers in the numerator go up instead of down. A 
slick way to prove it is to note that we want to count the number of integer solutions 
(a1,...,ak) to the relations 1 < a1 < a2 < -+> < ak < n. This is the same as 
0<aı <az2+1<---<ak+k-—1 < n+ k; and the number of solutions to 


0< bi < bo <- <b <n+k 


is the number of choices of k distinct things from the set {1, 2, ..., n+ k — 1}. (This 
trick is due to H. F. Scherk, Crelle 3 (1828), 97; curiously it was also given by W. A. 
Förstemann in the same journal, 13 (1835), 237, who said “One would almost believe 
this must have been known long ago, but I have found it nowhere, even though I have 
consulted many works in this regard.” ) 


61. If amn is the desired quantity, we have amn = Nam(n-1) + dmn by (46) and 
(47). Hence the answer is [n> m] n!/m!. The same formula is also easily obtained 
by inversion of (56). 


62. Use the identity of exercise 31, with (m, n, r, s, k) + (m+k, l—k, m+n, n+l, j): 
; l+k mt+tk/\n+k 
=u A m—k gan) 
l+k j l—-k-j m+l 
j,k 
7 l—j+k/(2l— 2j)! j! (m-l + j! (n+ji- D] 


by rearranging the factorial signs. The sum on k now vanishes unless j = l. 

The case | = m = n of this identity was published by A. C. Dixon [Messenger 
of Math. 20 (1891), 79-80], who established the general case twelve years later [Proc. 
London Math. Soc. 35 (1903), 285-289]. However, L. J. Rogers had already published 
a much more general formula in the meantime [Proc. London Math. Soc. 26 (1895), 
15-32, §8]. See also papers by P. A. MacMahon, Quarterly Journal of Pure and Applied 
Math. 33 (1902), 274-288, and John Dougall, Proc. Edinburgh Math. Society 25 
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(1907), 114-132. The corresponding g-nomial identities are 


BEEE 
- q\ n-k amn)? Nma ng 


qe 

k 

ee) eee) ee) are = See 
l+k kJa\ntk/a = manly 


where nla = [I (0 +g ++ q®). 
63. See CMath, exercises 5.83 and 5.106. 


64. Let f(n,m) be the number of partitions of {1,2,...,n} into m parts. Clearly 
f(1,m) = dim. If n > 1, the partitionings are of two varieties: (a) The element 
n alone forms a set of the partition; there are f(n — 1, m — 1) ways to construct 
partitions like this. (b) The element n appears together with another element; there 
are m ways to insert n into any m-partition of {1, 2, ..., n — 1}, hence there are 
mf(n — 1, m) ways to construct partitions like this. We conclude that f(n,m) = 
f(n—-1, m—-1)+mf(n—1, m), and f(n,m) = {7 } by induction. 


65. See AMM 99 (1992), 410-422. 
66. Let X = E), X = ( z ) = soa: X= (4) = =X, with similar notations 


n-1 n+l 
for Y and Z. We may assume that y > n — 1 is fixed, so that x is a function of z. 
Let F(z) = X — Y — Z, and suppose that F(z) = 0 for some z > n — 2. We will 
prove that F’(z) < 0; therefore z = y must be the only root > n—2, proving the second 


inequality. Since F(z) = “Sy (Y + Z) - 44 Y z="t+1Z = 0 and z > y and Y, Z > 0, 


n+l 
we must have Z=% < 2+". Setting X' = dX/dx and Z' = dZ/dz = dX/dz, we have 
V N O oe i 1 r l 1 je 
X x xz-1' 'g—-n+1~ ntl " z—-n+2 n+1 Z’ 
since =" < eet? Z} < Ž. Thus dz/dz = Z'/X' < *(Z/X), and 
F(z) = X dz z—-n,, Z contig < (2% ront a! <0. 
n+ldz n+l n n n+1 


To prove the first inequality, we may assume that n > 2. Then if X = Y + Z for 
some z > n — 2, the second inequality tells us that z = y. 

References: L. Lovász, Combinatorial Problems and Exercises (1993), Problem 
13.31(a); R. M. Redheffer, AMM 103 (1996), 62-64. 


67. If k > 0, exercise 1.2.5-24 gives the slightly sharper (but less memorable) upper 
bounds (7%) = sel < nk/kl < 4(42)* < (25). The corresponding lower bound is 


(7) > (eee , which is less memorable (but often sharper) than (7) > (2)*. 
68. Let tk = r)a (1 — p)” tF; then tk — tet = (Z) a — p)” T} (k — np). So the 


stated sum is 


XO (tea — te) + JO (te — test) = 2t pnp) 
k<[np] k>[np| 
[De Moivre stated this identity in Miscellanea Analytica (1730), 101, in the case that 
np is an integer; H. Poincaré proved the general case in his Calcul des Probabilités 
(1896), 56-60. See P. Diaconis and S. Zabell, Statistical Science 6 (1991), 284-302, for 
the interesting history of this identity and for a variety of similar formulas.] 
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SECTION 1.2.7 
1. 0, 1, and 3/2. 


2. Replace each term 1/(2" + k) by the upper bound 1/2”. 

3. HG < Eocrem 2/2"; 2"~1/(2"~* — 1) is an upper bound. 

A. (b) and (c). 

5. 9.78760 60360 44382 ... 

6. Induction and Eq. 1.2.6-(46). 

7. T(m+1,n)-T(m,n)=1/(m+1)—1/(mn+1)—---—1/(mn+n) < 1/(m+1)— 
(1/(mn +n) +: +1/(mn +n)) = 1/(m + 1) — n/(mn +n) = 0. The maximum 


value occurs at m = n = 1, and the minimum is approached when m and n get very 
large. By Eq. (3) the greatest lower bound is y, which is never actually attained. A 
generalization of this result appears in AMM 70 (1963), 575-577. 

8. By Stirling’s approximation, ln n! is approximately (n + 3) Inn—n+lnV2z7; also 
X; Hk is approximately (n+1) Inn—n(1—7y)+(7+4); the difference is approximately 
yn + ł lnn + .158. 

9. —1/n. 

10. Break the left side into two sums; change k to k + 1 in the second sum. 
11. 2-—H,/n—1/n, for n > 0. 


12. 1.000... is correct to more than three hundred decimal places. 


13. Use induction as in the proof of Theorem A. Or use calculus: Differentiate with 
respect to x, also evaluate at x = 1. 


14. See Section 1.2.3, Example 2. The second sum is 4 (H241 — HÊ). 

15. 53-1 (1/9) X p=; He can be summed by formulas in the text; the answer comes to 
(n+ 1)H? — (2n+1)Hn + 2n. 

16. Hon-1 — 4Hn-1. 

17. First solution (elementary): Taking the denominator to be (p — 1)!, which is a 
multiple of the true denominator but not a multiple of p, we must show only that the 


corresponding numerator, (p — 1)!/1+ (p — 1)!/2+ ---+ (p — 1)!/(p — 1), is a multiple 
of p. Modulo p, (p — 1)!/k = (p — 1)! k’, where k’ can be determined by the relation 
kk’ mod p = 1. The set {1',2',...,(p— 1)’} is just the set {1, 2, ..., p — 1}; so the 
numerator is congruent to (p—1)!(1+2+---+p-—1)=0. 


Second solution (advanced): By exercise 4.6.2-6, we have x? = x? — x (modulo p); 
hence [z] = dkp — ôk1, by exercise 1.2.6-32. Now apply exercise 6. 

The numerator of H,-1 is in fact known to be a multiple of p? when p > 3; see 
Hardy and Wright, An Introduction to the Theory of Numbers, Section 7.8. 


18. If n = 2"m where m is odd, the sum equals 27¥ m1 /me where mı and mz are both 
odd. [AMM 67 (1960), 924—925.] 


19. Only n = 0, n = 1. For n > 2, let k = |lgn]. There is precisely one term whose 


denominator is 2", so 2*~'H,, — 4 is a sum of terms involving only odd primes in the 


2 
denominator. If H, were an integer, 25-1 H, — 4 


20. Expand the integrand term by term. See also AMM 69 (1962), 239, and an article 
by H. W. Gould, Mathematics Magazine 34 (1961), 317-321. 


2 
21: Hip- Hy. 


22. (n+1)(H2 — HÈ) —2n(Hn — 1). 


would have a denominator equal to 2. 
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23. I’(n4+1)/P(n4+1) = 1/n + I'(n)/I (n), since r(x +1) = «I(x). Hence Hn = 
ytI'(n+1)/C(n4+1). The function y(x) = I'(x)/T (x) = Hz-1 — y is called the psi 
function or the digamma function. Some values for rational x appear in Appendix A. 
24. It is 

r lim e-m? Il (1 i z) ee — lim s(x +1)... (x+n) l 


n— o0 k n—00 nen! 
k=1 


Note: The ee of Hn considered in the previous exercise is therefore equal 
to HL = Dix>ol(l/(k 1)" — 1/(k +1 + x)”), when r = 1; the same idea can be used 
for larger values of r. The infinite product converges for all complex zx. 

25. HO”) = Jaai H® and Ho? = He. so the identity generalizes (8). [See 
L. Euler, Novi Comment. Acad. Sci. Pet. 20 (1775), 140-186, §2.] 


SECTION 1.2.8 
1. After k months there are Fk+2 pairs, so the answer is Fi4 = 377 pairs. 


2. In(g'°°°/ V5) = 1000 In ¢ — } In 5 = 480.40711; logo Fiooo is 1/(In 10) times this, 
or 208.64; Fiooo is therefore a 209-digit number whose leading digit is 4. 


4. 0, 1, 5; afterwards Fn increases too fast. 

5. 0, 1, 12. 

6. Induction. (The equation holds for negative n also; see exercise 8.) 

7. If dis a proper divisor of n, Fa divides Fan. Now Fa is greater than one and less 
than F, provided d is greater than 2. The only nonprime number that has no proper 
factor greater than 2 is n = 4; Fy = 3 is the only exception. 

8. Fai =; Fo =i F_»=( 1)""'F, by induction on n. 

9. Not (15). The others are valid, by an inductive argument that proves something 
true for n — 1 assuming it true for n and greater. 


10. When n is even, it is greater; when n is odd, it is less. (See Eq. (14).) 

11. Induction; see exercise 9. This is a special case of exercise 13(a). 

12. If G(z) = So Fanz”, (1 — z — 2?°)G(z2) = z + Foz? + Fiz? +- = z + 2°G(z). Hence 
G(z) = G(z) + zG(z2)°; from Eq. (17) we find Fn = ((3n+ 3)/5) Fa — (n/5)Fn+1. 

13. (a) an = rFn-1 + sFn. (b) Since (bn+2 + c) = (bn+1 +c) + (bn +c), we may 
consider the new sequence b;, = bn + c. Applying part (a) to bh, we obtain the answer 
cFn-1 + (e+ 1)Fn — 


iias meata (2) BE E 
m m-—1 0 


15. Cn = tan + Ybn + (1 — x — y)Fn. 


16. Fn+1. Induction, and w = 1 on T +(! U a i E a 


)(2™ yt") — (x° = y")(a™ — y™) is 
) S HDE a lis (/5)?. 


S 
> 


17. In general, the quantity (x” 
equal to (xy)"(a™—"—* — ym" Na : 
18. It is Fonsi. 

19. Let u = cos 72°, v = cos36°. We have u = w? — 1; v = 1 — 2sin?18° = 1 — 2u?. 
Hence u +v = 2(v? — u’), ie., 1 = 2(v — u) = w — 4w? +2. We conclude that 
v= so. (Also u = so, sin 36° = 25g”, sin 72° = 151/442, Another 
interesting angle is a = arctan ¢ = T+ł arctan 4 , for which we have sina = 57 1/ 44! z 
cosa = 5714712.) 
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20. Foi, 
21. Multiply by 2? +a —1; the solution is (x”*1F41+2"7? Fy —a)/(x?+a—1). If the 
denominator is zero, x is 1/¢ or 1/4; then the solution is (n + 1 — x” Fy41)/(2x + 1). 


22. Fm+2n; set t = 2 in the next exercise. 

1 n k pk pn—k ym k pk pn-k pm 
23. — > ( ) Fi F; — F; F; 

V5 - k ($ nE G FE Fia ) 


= gO" (OF + Fea)” — G(R + Pi1)") = Frin 


24. Fa+ı (expand by cofactors in the first row). 

25. 2” 4/5 Fn = (1 + V5)" —(1- V5)”. 

26. By Fermat’s theorem, 2?~! = 1; now apply the previous exercise and exercise 
1.2.6-10(b). 


27. The statement is true if p = 2. Otherwise Fp—1 Fp+1 — F? = —1; hence, from the 
previous exercise and Fermat’s theorem, Fp-1Fp+1 = 0 (modulo p). Only one of these 
factors can be a multiple of p, since Fy41 = Fp + Fp-1. 


28. 6”. Note: The solution to the recurrence an4+1 = Aan + B”, ao = 0, is 
an = (A” — B”)/(A — B) if AF B, an = nA"! if A=B. 
29. (a2) Ge O Ge Ge Ge G Ge 


rPoOoOoCcooo 
ooococo 


1 
1 
1 
1 
1 
1 
1 


aeoow1wnnrr Oo 


40 60 40 8 1 
(b) follows from (6). [É. Lucas, Amer. J. Math. 1 (1878), 201-204.] 


30. We argue by induction on m, the statement being obvious when m = 1: 
m [(m=k)/2] pm-2 pm _ (7 = D [(m-k)/2] pm-2 _ 
—1 Fiap Fk = Fm —1 F =0. 
(a) =)! ) rie Pe = Fa Oy ) nee = 
m m= mia 
OPD EE E 
k 


= (—1)” Fn yo. ages a Fr”? =0. 
k 

(c) Since (—1)* Fn—n = Fk-1Fm — FeFm—1 and Fm 4 0, we conclude from (a) 
and (b) that >, (Y) (11) [070/21 Fm7? Fp =0. 

(d) Since Fnk = Fk-1Fn + FkFn+1 the result follows from (a) and (c). This 
result may also be proved in slightly more general form by using the q-nomial theorem 
of exercise 1.2.6-58. References: Dov Jarden, Recurring Sequences, 2nd ed. (Jerusalem, 
1966), 30-33; J. Riordan, Duke Math. J. 29 (1962), 5-12. 


31. Use exercises 8 and 11. 


32. Modulo Fn the Fibonacci sequence is 0,1,..., Fnr—1,0, Pn-1,-—Fn-2,.... 


33. Note that cosz = $(e* + e’*) = —i/2, for this particular z; then use the fact 
that sin(n + 1)z + sin(n — 1)z = 2sin nz cos z, for all z. 
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34. Prove that the only possible value for Fk, is the largest Fibonacci number less 
than or equal to n; hence n — Fp, is less than F;,,~1, and by induction there is a unique 
representation of n — Fk. The outline of this proof is quite similar to the proof of the 
unique factorization theorem. The Fibonacci number system is due to E. Zeckendorf 
[see Simon Stevin 29 (1952), 190-195; Bull. Soc. Royale des Sciences de Liège 41 (1972), 
179-182]; but Section 7.2.1.7 points out that it was implicitly known in 14th-century 
India. Generalizations are discussed in exercise 5.4.2—10 and in Section 7.1.3. 


35. See G. M. Bergman, Mathematics Magazine 31 (1957), 98-110. To represent x>0, 
find the largest k with 6" < x and represent x as ¢* plus the representation of x — ¢*. 

The representation of nonnegative integers can also be obtained from the following 
all-integer recursive rules, starting with the trivial representations of 0 and 1: Let 
Ln = O° +6" = Friit+Fn-1. The representation of Lo, +m for 0 < m < Len-1 and 
n > 1 is ¢?"+¢ 2" plus the representation of m. The representation of Lon+41 + m for 
0<m < Lon and n> 0 is ¢?"*1+ 6-2"? plus the representation of m — ¢~?", where 
the latter is obtained by applying the rule 6* — ¢*~77 = g*—-1 4 T? 4... 4 TIt, 
It turns out that all strings a of Os and 1s, such that a begins with 1 and has no 
adjacent 1s, occur to the left of the radix point in the representation of exactly one 
positive integer, except for the strings that end with 10?"1; the latter strings never 
occur in such representations. 


36. We may consider the infinite string So, since Sn for n > 1 consists of the first Fn 
letters of Sæ. There are no double a’s, no triple b’s. The string Sn contains Fn—2 a’s 
and F,,_1 b’s. If we express m — 1 in the Fibonacci number system as in exercise 34, 
the mth letter of Sæ is a if and only if kr = 2. The kth letter of Sx is b if and 
only if |(k +1)¢~'| — [kt] = 1; the number of b’s in the first k letters is therefore 
|(k + 1)¢"1]. Also, the kth letter is b if and only if k = |m@| for some positive 
integer m. This sequence was studied by John Bernoulli III in the 18th century, by 
A. A. Markov in the 19th, and by many other mathematicians since then; see K. B. 
Stolarsky, Canadian Math. Bull. 19 (1976), 473-482. 


37. [Fibonacci Quarterly 1 (December 1963), 9—-12.] Consider the Fibonacci number 
system of exercise 34; ifn = Fx, +---+ Fx, > 0 in that system, let u(n) = Fk. Also 
let (0) = co. We find that: (A) If n > 0, u(n — u(n)) > 2u(n). Proof: u(n — u(n)) = 
_, > Fr.42 > 2Fp, since kr > 2. (B) If0 < m < Fk, u(m) < 2( Fe —m). Proof: Let 
p(m) = Fy; m < Fei + Fe-3 +--+ + Fy4(e—1-3) moa = —Fj-14(6-1-3) moa2 + Fr < 
—} Fj + Fp. (C) E0 <m < p(n), u(n- p(n) +m) < 2(u(n) —m). Proof: This follows 
from (B). (D) f0 < m < u(n), u(n — m) < 2m. Proof: Set m = u(n) — m in (C). 
Now we will prove that if there are n chips, and if at most q may be taken in the 
next turn, there is a winning move if and only if u(n) < q. Proof: (a) If u(n) > q all 
moves leave a position n’, q’ with u(n’) < q’. [This follows from (D), above.] (b) If 
u(n) < q, we can either win on this move (if q > n) or we can make a move that leaves 
a position n’, q' with u(n’) > q. [This follows from (A) above: Our move is to take 
u(n) chips.] It can be seen that the set of all winning moves, if n = Fp, +---+ Fkn, 
is to remove Fk, +--+ + Fr,, for some j with 1 < j < r, provided that j = 1 or 
Fk- > 2(Fk; +--+ Fy,.). 

The Fibonacci representation of 1000 is 987+13; the only lucky move to force a vic- 
tory is to take 13 chips. The first player can always win unless n is a Fibonacci number. 

The solution to considerably more general games of this type has been obtained 
by A. Schwenk [Fibonacci Quarterly 8 (1970), 225-234]. 


39. (3" — (—2)")/5. 
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40. We prove, by induction on m, that f(n) = m for Fm < n < Fin4i: First, f(n) < 
max(1+ f(Fm), 2+ f(n— Fin)) =m. Second, if f(n) < m there is some k < n with 
1+ f(k) < m (hence k < Fm-1) and 2+ f(n— k) < m (hence n—k < Fm-2); but then 
n < Fm-1 + Fm-2. [Thus the Fibonacci trees defined in Section 6.2.1 minimize the 
maximum root-to-leaf cost when a right branch costs twice as much as a left branch. ] 


41. Fkipi + + Fk, 1 = on + (7! +---+G*") is an integer, and the parenthesized 
quantity lies between $? + 8” +--- =o '—1land 67+ 84+- = 7". Similarly, 
Foy 1 teet Fe = 6 n+(G" +---4+6*") = fon). [Such Fibonacci shifting is 


a convenient way to convert mentally between miles and kilometers; see CMath, §6.6.] 


42. [Fibonacci Quarterly 6 (1968), 235-244.] If such a representation exists, we have 


mMPy_1+ nF = Fhin + Peotn +++++ Fen (x) 


for all integers N; hence two different representations would contradict exercise 34. 

Conversely, we can prove the existence of such joint representations for all non- 
negative m and n by induction. But it is more interesting to use the previous exercise, 
and to prove that such joint representations exist for possibly negative integers m and n 
if and only if m + ọn > 0: Let N be large enough so that |m@N~! + nG%| < $7, 
and represent mF'n—1 + nFy as in (x). Then mFyn + nFw41 = ọ(mFn-1 + nFw) + 
(MEN! + n@%) = f(¢(mFy_1 + nFw)) = Fkn +e + Fk +N41, and it follows 
that (*) holds for all N. Now set N = 0 and N =1. 


SECTION 1.2.9 
1. 1/(1 — 2z) + 1/(1 — 32). 
2. It follows from (6), since (7) = n!/k!(n — k)!. 


3. G'(z) = In(1/(1 — z))/( — 2)? + 1/(1 — z)?. From this and the significance of 
G(z)/(1 — z), we have Ņ?2i Hy = nHn — n; this agrees with Eq. 1.2.7-(8). 


4. Put t=0. 
5. The coefficient of z* is, by (11) and (22), 


oD! {3.30 


Now apply Eqs. 1.2.6-(46) and 1.2.6—(52). (Or, differentiate and use 1.2.6—(46).) 


6. (In(1/(1 — z)))’; the derivative is twice the generating function for the harmonic 
numbers; the sum is therefore 2H, -1/n. 


8. 1/((1— z)(1 — 2”)(1 — 29)...). [This is historically one of the first applications 
of generating functions. For an interesting account of L. Euler’s eighteenth-century 
researches concerning this generating function, see G. Pólya, Induction and Analogy in 
Mathematics (Princeton: Princeton University Press, 1954), Chapter 6.] 

9. HST + STS. + £93 + £5193 + 454. 

10. G(z) = (14 212)...(1+ znz). Taking logarithms as in the derivation of Eq. (38), 
we have the same formulas except that (24) replaces (17), and the answer is exactly the 
same except that S2, S4, S6,... are replaced by — S2, — S4, — S6, .... We have e1 = Si, 
e2 = 557 — 4S2, e3 = ZS} — 59192 + $93, e4 = HST — ZSTS2+ £93 + Z51S3 isa. 
(See exercise 9.) The recurrence analogous to (39) is nen = Sien—-1 — S2en-2 +--+. 
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Note: The equations in this recurrence are called Newton’s identities, since they were 
first published in Isaac Newton’s Arithmetica Universalis (1707); see D. J. Struik’s 
Source Book in Mathematics (Harvard University Press, 1969), 94-95. 


11. Since $>; Smz”/m = InG(z) = 1, 5,(-1)* (hiz+hez?+---)*/k, the desired 


coefficient is (—1)*17*24 tm —lin(ky + ko +--+ km — 1)!/kı! ko!...km!. [Multiply by 
(—1)"—! to get the coefficient of eft eka, ..ekm when Sm is expressed in terms of the 
e’s of exercise 10. Albert Girard stated the formulas for S1, S2, S3, and S4 in terms 
of e1, e2, e3, and e4 near the end of his Invention Nouvelle en Algébre (Amsterdam: 


1629); this was the birth of the theory of symmetric functions.] 
n 
12. mmn mien __ ae a . 
D amv 2" D (7 Jore = SO + w)"2" = 1/0 - z- we) 
m,n>O0 m,n>0 n>0 


13. pe eT f(t) dt = (ao +++: + an)(e7®” — et) /s. Adding these expressions 
together for all n, we find Lf(s) = G(e~*)/s. 


14. See exercise 1.2.6—38. 


15. Gn(z) = Gn-1(z) + 2Gn—2(z) + dno, so we find H(w) = 1/(1 — w — zw”). Hence, 
ultimately, we find 


a = (YB GER) vra when z # —}; 


G,(—4) = (n + 1)/2” for n > 0. 


1— zti” 
16. Gnr(z) = (l+24 ot TEF ) . [Note the case r = 00.] 
n DOP erp epe ar 


w In(1/(1—z)) 


(Alternatively, write it as e and expand first by powers of w.) 


18. (a) For fixed n and varying r, the generating function is 
1 1 1 1 
Gn(z) = (14+ z)(14 2z)...(1+ nz) ~ H 1)( H 2) sss (+ +n) 
-gie 
k 


by Eq. (27). Hence the answer is [ 


n+l 
n+1—-r. 


1 1 1 Ry as: 
l—-z 1-22 `` 1—nz wt at? 


by Eq. (28), so the answer is {"*"}. 


19. ,51(1/n—1/(n + p/q))a?*"4 = Drow"? In(1 — wa) — z” In(1 — z) + Sa? = 
f(x) + g(x), where w = e2™/4 and 


]. (b) Similarly, the generating function is 


f(x) = yo In(l—w*x), g(x) = (1—2?)In(1— x) 4 me x? In i 


k=1 
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Now limz-41~ g(x) = q/p — lnq. From the identity 


; ; 10/2 „—ið/2 
In(1 — etf) = In Cae £ ) = n24 


2i 


we may write limz—=1- f(x) = f(1) = A+ B where 


L in ik i j 
A= “(In2 7+") = nae peA 
oe? Meee nev DT (PI)? 
k=1 
= k k 
B= w *? Insin —7 = 5 (a 9 + wP) In sin <a 
k=1 q 0<k<q/2 q 
2pk k 
=2 5 cos eer Insin £r. 
0<k<q/2 q 
Finally, 
i, iG fl tu?) ifto? 1 P 
2° w-P-1) 2\i—-w 2 \ wr/? — WP? gr a 


[Gauss derived these results in §33 of his monograph on hypergeometric series, Eq. [75], 
but with insufficient proof; Abel provided a justification in Crelle 1 (1826), 314-315.] 
20. Cmk = kI{7}, by Eq. 1.2.6-(45). 

21. We find z?G’(z) + zG(z) = G(z) — 1. The solution to this differential equation is 
G(z) = (-1/z)e7!/*(Ei(-1/z) + C), where Ei(z) = JZ e™* dt/t and C is a constant. 
This function is very ill-behaved in the neighborhood of z = 0, and G(z) has no power 
series expansion. Indeed, since Vn! ~ n/e is not bounded, the generating function 
does not converge in this case; it is, however, an asymptotic expansion for the stated 
function, when z < 0. [See K. Knopp, Infinite Sequences and Series (Dover, 1956), 
Section 66.] 

22. Giz) = (14+ 2)" 427)" 4 7)" + 28)"... = (1—z)~”. It follows that the 
stated sum is Cee). 

23. (a) When m = 1 this is the binomial theorem, with fı(z) = z and gi(z) =1+4+z. 
When m > 1 we can increase m by 1 if we replace zm by zm(1 + zmij1) and let 


fm4i(21,-+-)%m41) = dm t fra seesamin + ani), Gmaili +++, 2m+41) = 
Zm+19m(2Z1;+++;%m—1;%m(1 + ami): Thus go(21, 22) = 21 + 22 + 2122 and 
Öm ies Zm) ae a 
fin Zigeeg tin) z3" 
1+ 
1+ 
Lin" 


Both polynomials fm and gm satisfy the same recurrence fm = Zm fm-1 + Zm-1fm-2, 
Jm = Zm9m—1+2%m-19m—2, With the initial conditions f_1 = 0, fo = g-1 = go = z0 = 1. 
It follows that gm is the sum of all terms obtainable by starting with z1...z, and 
striking out zero or more nonadjacent factors; there are Fm+2 ways to do this. A 
similar interpretation applies to fm, except that z; must remain. In part (b) we will 
encounter the polynomial hm = Zmgm-—1 + Zm—1fm-2; this is the sum of all terms 
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obtained from z1...2m by striking out factors that are not cyclically adjacent. For 
example, h3 = 212223 + 2122 + 2123 + 2223. 


(b) By part (a), Sn(21,..-,%m—1,2) = [2m] SO" 22 fin "Gm hence 


Sn(Z1,--+;2%m) < DD (") (".") a” > be g go 


O<s<r<n 


where a = Zm9m-—1, b = 2m-—1Gm—2, € = 2mfm-1, d = Zm-1fm-—2. Multiplying this 
equation by z” and summing first on n, then on r, then on s, yields the closed form 


1 p n+l 


—0 


Sn(21; -+ -3 2m) = [2"] (1 — az)(1 — dz) — bez? p—o 


where 1 — (a+ d)z + (ad — be)z? = (1 — pz)(1— oz). Here a +d = hm, and ad — be 
simplifies to (—1)™z1 ... Zm. [We have, incidentally, established the recurrence Sn = 
hmSn-1 — (—1)"21...2mSn-2, a relation that is not easy to derive without the help 
of generating functions.] 

(c) Let pı = (z + Vz? +4z)/2 and o1 = (z — Vz? +4z)/2 be the roots when 
m = 1; then pm = pt and om = of. 

Carlitz used this result to deduce a surprising fact: The characteristic polynomial 
det(xI — A) of the n x n matrix 


n=i not ea a 

Co) Cr) - Go) Go) 
of “right justified binomial coefficients” is }7, Ce (-1)!-)/21¢" with Fibonomial 
coefficients (see exercise 1.2.8-30). He also showed, using similar methods, that 


5 kı + k2 k2 + k3 km + kı ht zm 
kes ko sae kn AEP o 


k1,...5km >0 


ae 2, hm(=z71,..., BY? — 421... Zm 


[Collectanea Math. 27 (1965), 281-296.] 


24. Both sides are equal to >, (7) [2"](zG(z))*. When G(z) = 1/(1— 2), the identity 
becomes D>, (7) ("—,) = (+771), a case of 1.2.6-(21). When G(z) = (eë — 1)/z, it 


becomes >, m£{?} = m”, Eq. 1.2.6-(45). 


25. Yo ,[w*](1—2w)” [2"] 2" (1+2)? = [2"] (1+2) Yo, lw] (L—2w)"(2/(1+2)’)*, 
which equals [2°] (1 + z)?"(1 — 2z/(1 + z)?)” = [z"](1 + 27)” = (72) [n even]. Sim- 
ilarly, we find X, (Ẹ ye 2E) (= aye = C1)" aid Many examples of this summation 


technique can be found in G. P. Egorychev’s book Integral Representation and the 
Computation of Combinatorial Sums (Amer. Math. Soc., 1984), translated from the 
Russian edition of 1977. 


26. [F(z)| G(z) denotes the constant term of F(z~')G(z). See the discussion by D. E. 
Knuth in A Classical Mind (Prentice-Hall, 1994), 247-258. 
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SECTION 1.2.10 

1. Gn(0) = 1/n; this is the probability that X[n] is the largest. 

2. G"(1) = X, k(k — 1)pr, G'(1) = Do, kpr. 

3. (min 0, ave 6.49, max 999, dev 2.42). Note that HÊ is approximately 1/6; see 
Eq. 1.2.7—(7). 

4. (ppa. 

5. The mean is 36/5 = 7.2; the standard deviation is 6\/2/5 ~ 1.697. 

6. For (18), the formula 
In(q+ pe’) in(1 + pt 4 x 2i H =) = pt+ p(1 pË Hp(1— p)(1 2p) É Hirs 


tells us that k3/n = p(1 — p)(1 — 2p) = pq(q — p). (This nice pattern does not continue 
to the coefficient of t*.) Setting p = k™* gives us k3 = ÐP}, k (1 — k™')(1— 2k") = 
Hn — 3H© + 2H) in the case of distribution (8). And for (20), we have lIn G(e*) = 
t+ H(nt) — H(t) where H(t) = In((e’ — 1)/t). Since H’(t) = e*/ (e* — 1) —1/t, we have 
kr = (n" —1)B,/r for all r > 2 in this case; in particular, «3 = 0. 


7. The probability that A = k is pmx. For we may consider the values to be 
1,2,...,m. Given any partitioning of the n positions into m disjoint sets, there are m! 
ways to assign the numbers 1,...,m to these sets. Algorithm M treats these values as 
if only the rightmost element of each set were present; sO Pmk is the average for any 
fixed partitioning. For example, if n = 5, m = 3, one partition is 


{XH X14} {XP}, Xh {XB]; 


the arrangements possible are 12312, 13213, 21321, 23123, 31231, 32132. In every 
partition we get the same percentage of arrangements with A = k. 

On the other hand, the probability distribution does change if more information 
is given. If n = 3 and m = 2, for example, our argument in the previous paragraph 
considers the six possibilities 122, 212, 221, 211, 121, 112; if we know that there are 
two 2s and one 1, then only the first three of these possibilities should be considered. 
But this interpretation is not consistent with the statement of the exercise. 


8. M®/M”. The larger M is, the closer this probability gets to one. 


9. Let qnm be the probability that exactly m distinct values occur; then from the 
recurrence 
_M-m+i1 a m 
qnm = M d(n—=1)(m—1) mM iem 


we deduce that 
acs =mi{ ni }/ M- mim”. 
m 
See also exercise 1.2.6-64. 


10. This is qnmPmk summed over all m, namely M~" X n (5) Jo leal There does 
not appear to be a simple formula for the average, which is one less than 


M fi n 
Hu- >) (1- Ym = nt w - 1) Bp MEK. 
m=1 k=1 


11. Since this is a product, we add the semi-invariants of each term. If H(z) = z 
H(e') = e™, so we find kı = n and all others are zero. Therefore, mean(F) = n + 
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mean(G), and all other semi-invariants are unchanged. (This accounts for the name 
“semi-invariant.” ) 


12. The first identity is obvious by writing out the power series for e*’. For the second, 
let u = 1 + Mıt + Mot?/2! +--+: when t = 0 we have u = 1 and Dku = Mx. Also, 
Di (nu) = (-1)~*(j — 1)!/u’. By exercise 11, the same formula applies for central 
moments except that we leave out all terms with ki > 0; thus k2 = m2, k3 = Msz, 
K4 = M4 — 3m2. 
waga o ee (14 zya OMH) =—(1+0(n™) 
5 I'(z+1)n! I'(z+1) n ! I'(z+1) ; 
Let zn = e/°". When n > œ and t is fixed, we have zn — 1; hence (zn +1) > 1, 
and 


n7! 


lim z;""Gn(zn) = lim exp (= p (e/a — 1) mn) 


n—0o n— oo On 


= lim exp (= H o( : )) Lg 

n= 203 | v logn l 
Notes: This is a theorem of Goncharov [Izv. Akad. Nauk SSSR Ser. Math. 8 (1944), 
3-48]. P. Flajolet and M. Soria [Disc. Math. 114 (1993), 159-180] have extended the 
analysis to show that Gn(z) and a large family of related distributions not only are 


approximately normal near their mean values, they also have uniformly exponential 
tails, in the sense that 


for some positive constant a and for all n and z. 

14. e7tPn/ VPI" (q 4 pett/VPan)n — (ge—ttP/VPaM + petta/ vean)”, Expand the exponen- 
tials in power series, to get (1—t?/2n+O(n7*/?))” = exp(nIn(1—t?/2n+0(n™?/?))) = 
exp(—t?/2 + O(n™™?)) > exp(—t?/2). 


15. (a) Vaso e~” (uz) */k! = e#@-Y, (b) Ine”(*-D = u(e* — 1), so all semi-invariants 


oe ‘a exp(—itnp//np ) exp(np(it//np — t?/(2np) + O(n~*/?))) = exp(—t?/2 + 
O(n-'/*)), 


16. g(z) = 20), Pege(z); mean(g) = >`, Pe mean(gx); and var(g) = >%), Pr var(ge) + 
Y jen Pipk(mean(g;) — mean(gx))?. 
17. (a) The coefficients of f(z) and g(z) are nonnegative, and f(1) = g(1) = 1. Clearly 
h(z) shares these same characteristics, since h(1) = g(f(1)), and the coefficients of h 
are polynomials in those of f and g, with nonnegative coefficients. 

(b) Let f(z) = Y` pz” where px is the probability that some event yields a “score” 
of k. Let g(z) = Si qz? where qx is the probability that the event described by f 
happens exactly k times (each occurrence of the event being independent of the others). 
Then A(z) = So rz", where rz is the probability that the sum of the scores of the events 
that occurred is equal to k. (This is easy to see if we observe that f(z)” = Ð st2*, 
where s+ is the probability that a total score t is obtained in k independent occurrences 
of the event.) Example: If f gives the probabilities that a woman has k female offspring, 
and if g gives the probabilities that there are k females in the nth generation, then h 
gives the probabilities that there are k females in the (n + 1)st generation, assuming 
independence. 

(c) mean(h) = mean(g) mean( f); var(h) = var(g) mean? (f) + mean(q) var( f). 
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18. Consider the choice of X[1],...,X[n] as a process in which we first place all the 
n’s, then place all the (n — 1)’s among these n’s, ..., finally place the ones among 
the rest. As we place the r’s among the numbers {r+ 1,...,7}, the number of local 
maxima from right to left increases by one if and only if we put an r at the extreme 
right. This happens with probability kr/(kr + kr4i +--+- + kn). 


19. Let a, = l. Then a, is a left-to-right maximum of a1 ...a, <> j < kimplies a; < l 
<=> aj > l implies j > k — > j > l implies bj > k 4> k is a right-to-left minimum of 
by... bn. 


20. We have mz = max{ai — bi,...,@n — bn}. Proof: If not, let k be the smallest 
subscript such that az — bk > my. Then ax is not a left-to-right maximum, so there is 
aj < k with aj > ap. But then a; — bj > a, — bk > mz, contradicting the minimality 
of k. Similarly, mr = max{bi—a1,...,bn—an}. 


21. The result is trivial when € > q, so we may assume that e < q. Setting x = pts qe 


in (25) gives Pr(X > n(pte)) < (Cac ee Now (see)? < e-* since 
i 3 


t < e*™! for all real t. And (q—«)n 74 = € Heg- gq? <e- ne. 
(A more detailed analysis yields the slightly stronger estimate exp(—e?n/(2pq)) when 
p > 4; still further work yields the upper bound exp(—2e’n) for all p.) 


By reversing the roles of heads and tails we find 


Pr(X < n(p—)) =Pr(n— X > n(qte)) seo). 
(One should not confuse “tails” with the tail of a probability distribution.) 


22. (a) Set x =r in (24) and (25), and note that qx + per = 1 + (r — 1)pp < e~ YP, 
[See H. Chernoff, Annals of Math. Stat. 23 (1952), 493-507.] 

(b) Let r =1+6 where |5| < 1. Then r~"e”~! = exp( 
is < e-8°/2 when 6 < 0 and < e-4/3 when 6 > 0. 

(c) The function rtel“!/" decreases from 1 to 0 as r increases from 1 to oo. If 
r > 2 it is < e1/7/2 < .825; if r > 4.32 it is 1/2. 

Incidentally, the tail inequalities with « = r give precisely the same estimate 
(r—"e”—')* when X has the Poisson distribution of exercise 15. 


23. Setting « = #3% in (24) gives Pr(X < n(p—e)) < (P (PTI < 


P 
e7®n/ (ra). Similarly, £ = pee 1 yields Pr(X > n(p + €)) < ( 


Let f(e) = (q+e) In(1+ $)—(p+e)In(1+ $), and note that f’(e) = 


q 
It follows that f(e) < —e?/(6pq) if 0 < € < p. 


340 +350 — oo), which 


SECTION 1.2.11.1 
1. Zero. 


2. Each O-symbol represents a different approximate quantity; since the left-hand 
side might be f(n) — (—f(n)) = 2f(n), the best we can say is O(f(n)) — O(f(n)) = 
O(f(n)), which follows from (6) and (7). To prove (7), note that if |zn| < M|f(n)| for 
n > no and |z| < M"|f(n)| for n > no, then jan xh) < |en] + lxh] < (M+ M’)|f(n)| 
for n > max(no, no). (Signed, J. H. Quick, student.) 

3. n(Inn) +yn+4+ O(./nInn). 

4. Ina + (Ina)*/2n + (Ina)*/6n? + O(n™?). 

5. If f(n) = n? and g(n) = 1, then n belongs to the set O( f(n) + g(n)) but not to 
the set f(n) + O(g(n)). So the statement is false. 
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6. A variable number, n, of O-symbols has been replaced by a single O-symbol, falsely 
implying that a single value of M will suffice for each term |kn| < Mn. The given sum is 
actually @(n*), as we know. The last equality, >?_, O(n) = O(n?), is perfectly valid. 

7. If x is positive, the power series 1.2.9-(22) tells us that e” > #"*1/(m+1)!; hence 
the ratio of e*/x™ is unbounded by any M. 

8. Replace n by e” and apply the method of the previous exercise. 

9. If |f(z)| < M|z|™ for |z| < r, then Jef] < eM" = 1 + |z|™(M + M?|2/™/2! + 
M3 |z/?™/3) +) <1 4 fz|™(M + M?r™/2! + M3 r?/3! +--+), 

10. In(1 + O(z™)) = O(z”), if m is a positive integer. Proof: If f(z) = O(z™), there 
exist positive numbers r < 1, r’ <1, and a constant M such that |f(z)| < M|z|™ <r’ 
when |z| < r. Then |In(1 + f(z))| < |f(z)| + PHON +- < ||” M(1+ Ly! +e). 
11. We can apply Eq. (12) with m = land z = ln n/n. This is justified since ln n/n < r 
for any given r > 0, when n is sufficiently large. 


12. Let f(z) = (ze*/(e* —1))"?. Tf Pe were O(n”), the stated identity would show 
that [z"] f(z) = O(n*/(k—1)!), so f(z) would converge when z = 2ri. But f(27i) = 00. 


13. Proof: We may take L = 1/M in the definitions of O and 2. 


SECTION 1.2.11.2 

1. (Bo+Biz+ Boz?/2!+---)e*=(Bot+ Bizt Boz?/2!4+---)+2; apply Eq. 1.2.9-(11). 

2. The function Bm4+i({x}) must be continuous, for the integration by parts. 

3. |Rmn| < |Bal (| fE |f°™ (a)| da. [Notes: We have Bm(£) = (—1)” Bm(1 — £), 
and B,,(x) is m! times the coefficient of z” in ze**/(e* — 1). In particular, since 
e*/?/(e* — 1) = 1/(e?/? — 1) — 1/(e7 — 1) we have Bm(4) = (2*~™ — 1)Bm. It is not 
difficult to prove that the maximum of |Bm—Bm(z)| for 0 < x < 1 occurs at x = 5 when 
m is even. Now when m = 2k > 4, let us write simply Rm and Cm for the quantities 
Rmn and Cmn. We have Rm—2 = Cm + Rm = f? (Bm — Bm({x}))f™ (x) dx/ml, 
and Bm — Bm({x}) is between 0 and (2 — 2'~)Bm; hence Rm-—2 lies between 0 and 
(2 — OPA) Cr It follows that Rm lies between —Cm and (1 — 217) Cm, a slightly 
stronger result. According to this argument we see that if f™+? (x) f™+9 (x) > 0 
for 1 < x < n, the quantities Cm+2 and Cm+4 have opposite signs, while Rm has the 
sign of Cm+2 and Rm+2 has the sign of Cm+4 and |Rm+2| < |Cm+2|; this proves (13). 
See J. F. Steffensen, Interpolation (Baltimore: 1927), §14.] 


mH AB ! 1 1 
4. = ke _ n k m nik = —— Bm4 (n)-—— Baia 


ee 2 wet). 9 ALA pet 
= 1 =4 =27. 
£ = abon (n—4)2(n— i)... G) 1.3.3.5.. f 
6. Assume that c > 0 and consider ` o<p<n n(k + c). We find 


In(c(c +1)... (c+n— 1)) = (n+ c)ln(n +c) — cnc -n — į n(n+c)+ nc 
1 B,(—-1)* 1 1 | 
+ > k(k —1) Cea =r) + Rmn. 


1<k<m 
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Also 
B,(—1)* Bal Lieka 
!— 1 fi I 
In(n—1)!=(n— 5) Inn-n+o4 kk = 1) ma IF 


Now ln In-1(c) = cln(n— 1) +In(n—1)!—In(c... (e +n—1)); substituting and letting 
n — œ, we get 


© Bm({x ) da 
ln P(e) = —c+ (c— 3) Inc+o4 5 ik re = ae 


1<k<m 


This shows that P(c + 1) = ce?) has the same expansion we derived for cl. 


. An /2+n/2+1/12¢-n7/4 where A is a constant. To obtain this result, apply Euler’s 
a formula to )77— = klnk. A more accurate formula is obtained if we multiply 
the answer above by 


exp(—Ba/(2-3-4n?) —--- — Bot/((2t — 2)(2t — 1)(2t)n”*~?) + O(1/n”*)). 


In these formulas, A is the “Kinkelin—Glaisher constant” 1.2824271... [Crelle 57 
(1860), 122-158; Messenger of Math. 7 (1877), 43-47], which can be shown to equal 
el/12—0(-1) = (Qre?— 6 (2)/6(2) 1/12 [de Bruijn, Asymptotic Methods in Analysis, §3.7]. 

8. We have, for example, In(an? + bn) = 2Inn +Ina+In(1 + 6/(an)). Thus the 
answer to the first question is found to be 2an? Inn +a(Ina—1)n?+2bnInn+bnIna+ 
Inn + b?/(2a) + $ lna +o + (3a — b’)b/(6a?n) + O(n~7). Massive cancellation occurs 
when we compute the quantity In (en)! — In (en? — n)!—nIne—Inn?!4 In (n? — n)! = 
(e — 1)/(2c) — (e — 1)(2c — 1) /(6c?n) + O(n~?). The answer is therefore 


e/09) (1 e (1 k O(n?) 


6c2n 


Incidentally, Ce ie CN can be written | J} 1 (1+aj/(n? — j)) where a= 1 — 1/c. 
H 1 4 O(n™?), and In (m1)? = 


9. (a) We have ln (2n)! = (2n + > 2n — 2n +04 zin | 
(2n + 1)lnn — 2n + 20 + <4 O(n-3 ; hence AG ), = exp(2nIn2 — $Inan— f+ 
O(n-3)) = 2?” (nn)? (1 int 4 = H =3)). (b) Since er) = pem oa) 


and Ca) = I'(n+1/2)/(nI'(n)IP'(1/2)) = Ree E we obtain the same result 
from 1.2.11.1—(16) because 


1/2 1/2 1/2 1 1/2 1/2 3/2 1 
He LACO- LACA- 
1/2 ol 2 8’ |-3/2 4 4 128 
Method (b) explains why the denominators in 
a _ a nt n? Bn? 2n 8399n 869E (n7) 
yrn 8 128 ' 1024 32768 262144 © l 
are all powers of 2 [Knuth and Vardi, AMM 97 (1990), 629-630]. 


n 4194304 


SECTION 1.2.11.3 
1. Integrate by parts. 
2. Substitute the series for e~* in the integral. 
3. See Eq. 1.2.9-(11) and exercise 1.2.6—48. 
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4. 1 + 1/u is bounded as a function of v, since it goes to zero as v goes from r to 
infinity. Replace it by M and the resulting integral is Me". 


5. f'(x) = f(x)((n + 1/2)(n — 1/2) /a? — (2n + 1)/a +1) changes sign at the point 
r=n+1/2— y/n + 1/2, so |R| = of |f" (x)| dx) = of f” (x) dx — Je f(x) dz) = 
O(f'(n) — 2f"(r) + f'(0)) = Of (m)/Vn). 

6. It is n"+% exp((n + B)(a/n — a?/2n? + O(n-%))), ete. 

7. The integrand as a power series in z™* has the coefficient of x™” as O(u?"). After 
integration, terms in 2° are Cu"/x? = O(a~°/4), etc. To get O(x7°?) in the answer, 
we can discard terms u”/r™” with 4m — n > 9. Thus, an expansion of the product 
exp(—u?/22x) exp(u3/3x7)... leads ultimately to the answer 


3 5 4 7 6 9 5 
ya" Y 1/4, Y p34 Y p Y p54 3/2 -( y y a O(a), 


6 "40 "12% ~ 336" 36 (3456 20 

8. (Solution by Miklós Simonovits.) We have |f(x)| < x if x is large enough. Let 
R(x) = Ce —e~'2)) du be the difference between the two given integrals, 
where g(u, x) = u—aln(1+u/2) and h(u, £) = u?/2a—u8/3a? +---+(-1)™u™/ ma) 
Notice that g(u,z) > 0 and h(u,x) > 0 when |u| < a; also g(u,xz) = h(u,x) + 
O(u™ t/a"). 

According to the mean value theorem, e° — e? = (a — b) e° for some c between 

a and b. Therefore |e* — e°| < |a — b| when a,b < 0. It follows that 


|\f(z)| Mz” u™tlduy 
|R(a)| a) ote) — nua) du = of f a 


LF (@)| -Mar 2™ 


= O(g)r—™) = O(a~°). 


9. We may assume that p Æ 1, since p = 1 is given by Theorem A. We also assume 
that p Æ 0, since the case p = 0 is trivial. 
Case 1: p <1. Substitute t = px(1 — u) and then v = —In(1 — u) — pu. We have 
dv = ((1 — p + pu)/(1 — u)) du, so the transformation is monotone for 0 < u < 1, and 
we obtain an integral of the form 


a =e ( l-u ) 
xe ~“ dv| ————— _}. 
o 1- p+ pu 


Since the parenthesized quantity is (1 — p)™+(1 — v(1 — p)~? +---), the answer is 
p 1l-pyx e "x? 1 —2 
—- 1 + : 
pee Y Hey l gaa te) 


Case 2: p > 1. This is 1 — So ). In the latter integral, substitute t = px(1+ u), 
then v = pu — ln(1 + u), and proceed as in Case 1. The answer turns out to be the 
same formula as Case 1, plus one. Notice that pet™? < 1, so (pe!~?)® is very small. 

The answer to exercise 11 gives another way to solve this problem. 


Pp l-p\x -a „£ -y e ¥(e¥ -1- y — y?/2) —2 
10. —— 1 t : 
ot re wa ( e ap iF O(a ~*) 
11. First, rQ2(n) + Rijen) = n! (a/n)"e"/® generalizes (4). Next, we have R,(n) = 
n! (e"/nx)" y(n, nx)/(n —1)!, generalizing (9). Since ay(a,x) = y(a+1, x) +e "2" we 
can also write Rz(n) = 1+ (e*/nz)"y(n + 1, nx), relating this problem to exercise 9. 
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Moreover, we can tackle Q,(n) and R,(n) directly by using Eqs. 1.2.9-(27) and (28) 
to derive series expansions involving Stirling numbers: 


1+2Q2(n) =X 2*n* n? = 5 Se 


k>0 kym>0 
Ren =X rn" (n4 i= oz = ee 
k>0 kym>0 


The sums over k are convergent for fixed m when |x| < 1, and when |x| > 1 we can use 
the relation between Q2(n) and R1/2(n); this leads to the formulas 


—_ 1 r (=1)"am(z) zim 
(n) l—x (1—2z)3n (l—2x)2mrtinm O(n J; 
— £! z (=1)"rm(2) siema .< 
R(n) Lea ean 0an O(n ), ifa<1; 
ie E Po C1 aml@) | op -im 
Qz(n) n” "l-2 (l=x)n | (=r) tine | Oln )s 
ome” 1 x o C1" rm(a) airh 
Ry n) nan T E -rn T T (1 — x)2"Finm T O( ), ife>l. 
Here 
_ ffm 2m—1 m 2m—2 4) 
am(e) = (pom + CT rr 
and 


mt = (Bye (Toten 


are polynomials whose coefficients are “second-order Eulerian numbers” [CMath §6.2; 
see L. Carlitz, Proc. Amer. Math. Soc. 16 (1965), 248-252]. The case x = —1 is 
somewhat delicate, but it can be handled by continuity, because the bound implied by 
O(n~'~™) is independent of x when x < 0. It is interesting to note that R-1(n) — 
Q-1(n) = (-1)"n!/(e"n”) & (—1)" V2rn/e?” is extremely small. 

12. (3, 32")/v2. 

13. See P. Flajolet, P. Grabner, P. Kirschenhofer, and H. Prodinger, J. Computational 
and Applied Math. 58 (1995), 103-116. 


15. Expanding the integrand by the binomial theorem, we obtain 1 + Q(n). 
16. Write Q(k) as a sum, and interchange the order of summation using Eq. 1.2.6—(53). 


17. S(n) = yrn/2 + 2 -— BWV n/2n -— Fen) + BV 7/23 + O(n’). [Note that 
S(n+1) + P(n) = Vise k”-¥k!/n!, while Q(n) + R(n) = edo M/k! ne* 


18. Let Sn(x,y) =o, (Z) (Œ + k)" (y +n — k)”. Then for n > 0 we have S(x,y) = 
ty, (K) (E +k) (y +n k tnd, CR )\(etl+k)*ytn-1—kye t= 
(x+y +n)” +nSn-1(£ + 1, y) by Abel’s formula 1.2.6-(16); consequently Sn(x, y) = 
E, (ke +y + n)”. [This formula is due to Cauchy, who proved it using the 
calculus of residues; see his Œuvres (2) 6, 62—73.] The stated sums are therefore equal 
respectively to n”(1 + Q(n)) and (n+ 1)"Q(n +1). 
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19. sae Cn exists for all n > N and |f(z)| < Mz* for 0 < x < r. Let F(x) = 
Se -Nt F(t) dt. Then when n > N we have 


[Cn <f e-*|F(a)| dx + 
0 


f e7 0M2 e N? F(x) dx 


f e7")? F(x) dx 


<u f e "a dx + (n— N) sup|F(@)| f e ON) dg 
0 r 


2>r 


<M ['e “re dx + (n— N) 
o 


=MI(a + 1)n7™® + sup| F(2)| e7 "0" = O(n"). 


z>r 
[E. W. Barnes, Phil. Trans. A206 (1906), 249-297; G. N. Watson, Proc. London Math. 
Soc. 17 (1918), 116-148.] 

20. [C. C. Rousseau, Applied Math. Letters 2 (1989), 159-161.] We have Q(n) +1 = 
nj, e "(+2)" de = n f e Me—In(1+2)) qyp = n foe "“g(u) du, by substituting 
U=2£ a) +x) and letting g(u) = dx/du. Notice that x = 77°, cx(2u)*/? when u is 
sufficiently small. Hence g(u) = PTL kex(2u)*/?-! + O(u™/?-1), and we can apply 
Watson’s lemma to Q(n) + 1—n f e7"™ yo hex (Qu)*/?4 du. 


SECTION 1.3.1 

1. Four; each byte would then contain 34 = 81 different values. 

2. Five, since five bytes is always adequate but four is not. 

3. (0:2); (3:3); (4:4); (5:5). 

4. Presumably index register 4 contains a value greater than or equal to 2000, so that 
a valid memory address results after indexing. 


5. ‘DIV -80,3(0:5)’ or simply ‘DIV -80,3’. 


6. (a) rA —|-]5] 1] 200 [15]. (b) rl2 + —200. (c) XH ]+]ololslil? 


(d) Undefined; we can’t load such a big value into an index register. (e) rX © 


-/0;0)/0]0/0 


7. Let n = |rAX| be the magnitude of registers A and X before the operation, and 
let d = |V| be the magnitude of the divisor. After the operation the magnitude of rA is 
|n/d|, and the magnitude of rX is n mod d. The sign of rX afterwards is the previous 
sign of rA; the sign of rA afterwards is + if the previous signs of rA and V were the 
same, otherwise it is —. 

Stating this another way: If the signs of rA and V are the same, rA + |rAX/V| 
and rX + rAX mod V. Otherwise rA + [rAX/V] and rX + rAX mod —V. 


8. rA¢/+]0}] 617/0}/1]}/;rX¥<]-]O]O]O}1] 14 


9. ADD, SUB, DIV, NUM, JOV, JNOV, INCA, DECA, INCX, DECK. 
10. CMPA, CMP1, CMP2, CMP3, CMP4, CMP5, CMP6, CMPX. (Also FCMP, for floating point.) 
11. MOVE, LD1, LD1N, INC1, DEC1, ENT1, ENN1. 
12. INC3 0,3. 
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13. ‘JOV 1000’ makes no difference except time. ‘JNOV 1001’ makes a different setting 
of rJ in most cases. ‘JNOV 1000’ makes an extraordinary difference, since it may lock 
the computer in an infinite loop. 


14. NOP with anything; ADD, SUB with F = (0:0) or with address equal to * (the 
location of the instruction) and F = (3:3); HLT (depending on how you interpret the 
statement of the exercise); any shift with address and index zero; SLC or SRC with 
index 0 and address a multiple of 10; MOVE with F = 0; STJ *(0:0), STZ *(0:0), and 
STZ *(3:3); JSJ *+1; any of the INC or DEC instructions with address and index zero. 
But ‘ENT1 0,1’ is not always a no-op, because it might change rll from —0 to +0. 


15. 70; 80; 120. (The block size times 5.) 


16. (a) STZ 0; ENT1 1; MOVE 0(49); MOVE 0(50). If the byte size were known to equal 
100, only one MOVE instruction would have been necessary, but we are not allowed to 
make assumptions about the byte size. (b) Use 100 ST2’s. 


17. (a) STZ 0,2; DEC2 1; J2NN 3000. 


(b) STZ 0 
ENT1 1 
JMP 3004 

(3003) MOVE 0(63) 

(3004) DEC2 63 


J2P 3003 
INC2 63 
ST2 3008(4:4) 
(3008) MOVE 0 I 
(A slightly faster, but quite preposterous, program uses 993 STZ’s: JMP 3995; STZ 1,2; 
STZ 2,2; ...; STZ 993,2; J2N 3999; DEC2 993; J2NN 3001; ENN1 0,2; JMP 3000, 1.) 


18. (If you have correctly followed the instructions, an overflow will occur on the ADD, 
with minus zero in register A afterwards.) Answer: Overflow is set on, comparison is 


set EQUAL, rA is set to | - |30|30|30|30|30| , rX is set to | - |31/30]30/30/30] , rI1 is 


set to +3, and memory locations 0001, 0002 are set to +0. (Unless the program itself 
begins in location 0000.) 


19. 42u = (2+1 +2+2+1+1+1+2+2+1+2+2+3+ 10+ 10)u. 
20. (Solution by H. Fukuoka.) 
(3991) ENT1 0 
MOVE 3995 (standard F for MOVE is 1) 
(3993) MOVE 0(43) (3999 = 93 times 43) 
JMP 3993 
(3995) HLT 0 I 


21. (a) Not unless it can be set to zero by external means (see the “GO button”, 
exercise 26), since a program can set rJ + N only by jumping from location N — 1. 


(b) LDA -1,4 


LDX 3004 
STX -1,4 
JMP -1,4 


(3004) JMP 3005 
(3005) STA -1,4 J 
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22. Minimum time: If b is the byte size, the assumption that |x] < b implies that 
X? < b, so X? can be contained in one byte. The following ingenious solution due to 
Y. N. Patt makes use of this fact. The sign of rA is the sign of X. 


(3000) LDA 2000 


MUL 2000(1:5) TA rX 
STX 3500(1:1) -———-- Aa 
SRC 1 X’ olololoļoloļloļolo 
MUL 3500 X* Jjololololololo]lo 
STA 3501 X* jololololololo]lo 
ADD 2000 X* lololxlol]oļlolo]lo 
MUL 3501(1:5) X8 ol Xë Jolo]lo 
STX 3501 x8 o| X5 lololo 
MUL 3501(1:5) | 0 xt oloļolo 
SLAX 1 x oloļoļolo 
HLT 0 

(3500) NOP 0O 

(3501) NOP O J 


space = 14; time = 54u, not counting the HLT. 


At least five multiplications are “necessary,” according to the theory developed in 


Section 4.6.3, yet this program uses only four! And in fact there is an even better 
solution below. 


Minimum space: (3000) ENT4 12 DEC4 1 
LDA 2000 J4P 3002 
(3002) MUL 2000 HLT 0 I 
SLAX 5 


space = 7; time = 171u. 
True minimum time: As R. W. Floyd points out, the conditions imply that |X| < 5, 
so the minimum execution time is achieved by referring to a table: 
(3000) LD1 2000 
LDA 3500,1 
HLT 0O 
(3495) (—5)*8 [This line needed only when b > 65.] 
(3496) (—4)'* 


(3505) (+5)*8 [This line needed only when b>65.] J 
space = 14; time = 4u. 


23. The following solution by R. D. Dixon appears to satisfy all the conditions: 


(3000) ENT1 4 DEC1 1 

(3001) LDA 200 JiNN 3001 
SRA 0,1 SLAX 5 
SRAX 1 HLT 0 J 


24. (a) DIV 3500, where 3500 =|+/1/0]0]0]0 
(b) SRC 4; SRA 1; SLC 5. 
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25. Some ideas: (a) Obvious things like faster memory, more input-output devices. (b) 
The I field could be used for J-register indexing, and/or multiple indexing (to specify 
two different index registers) and/or “indirect addressing” (exercises 2.2.2—3, 4, 5). 
(c) Index registers and J register could be extended to a full five bytes; therefore 
locations with higher addresses could be referred to only by indexing, but that would 
not be so intolerable if multiple indexing were available as in (b). (d) An interrupt 
capability could be added, using negative memory addresses as in exercise 1.4.4-18. 
(e) A “real time clock” could be added, in a negative memory address. (f) Bitwise 
operations, jumps on register even or odd, and binary shifts could be added to binary 
versions of MIX (see, for example, exercises 2.5-28, 5.2.2-12, and 6.3-9; also Program 
4.5.2B, 6.4-(24), and Section 7.1). (g) An “execute” command, meaning to perform 
the instruction at location M, could be another variant of C = 5. (h) Another variant 
of C = 48,...,55 could set CI + register: M. 


26. It is tempting to use a (2:5) field to get at columns 7-10 of the card, but this 
cannot be done since 2-8+ 5 = 21. To make the program easier to follow, it is 
presented here in symbolic language, anticipating Section 1.3.2. 


characters 
BUFF EQU 29 Buffer area is 0029-0044 punched on card: 
ORIG (0) 
00 LOC IN 16(16) Read in second card. u0u06 
01 READ IN BUFF (16) Read next card. uZu06 
02 LD1 0(0:0) rll + 0. uuuuI 
03 JBUS *(16) Wait for read to finish. uCu04 
04 LDA BUFF+1 rA + columns 6-10. 0,EH 
05 =1= SLA 1 uåuuF 
06 SRAX 6 rAX + columns 7-10. uF CF 
07 =30= NUM 30 LOWE 
08 STA LOC LOC + starting location. un JEU 
09 LDA BUFF+1 (1:1) uOuIH 
10 SUB =30=(0:2) uGuBB 
11 LOOP LD3 LOC rI3 + LOC. uuuEJ 
12 JAZ 0,3 Jump, if transfer card. uiCA: 
13 STA BUFF BUFF < count. UZ EU 
14 LDA LOC uuuEH 
15 ADD =1=(0:2) uE BA 
16 STA LOC LOC + LOC + 1. uuuEU 
17 LDA BUFF+3,1(5:5) U2A-H 
18 SUB =25=(0:2) US BB 
19 STA 0,3(0:0) Store the sign. uC YU 
20 LDA BUFF+2,1 1 AEH 
21 LDX BUFF+3, 1 u2AEN 
22 =25= NUM 25 uVuuE 
23 STA 0,3(1:5) Store the magnitude. uuCLU 
24 MOVE 0,1(2) rll ¢ rll +2. (!) uuABG 
25 LDA BUFF UZ EH 
26 SUB =1=(0:2) Decrease the count. UE, BB 
27 JAP LOOP Repeat until the count is zero. uJ uBs 


28 JMP READ Now read a new card. uå =f 
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SECTION 1.3.2 
1. ENTX 1000; STX X. 


2. The STJ instruction in line 03 resets this address. (It is conventional to denote 
the address of such instructions by ‘*’, both because it is simple to write, and because 
it provides a recognizable test of an error condition in a program, in case a subroutine 
has not been entered properly because of some oversight. Some people prefer ‘*-*’.) 


3. Read in 100 words from tape unit zero; exchange their maximum with the last of 
them; exchange the maximum of the remaining 99 with the last of those; etc. Eventually 
the 100 words will become completely sorted into nondecreasing order. The result is 
then written onto tape unit one. (Compare with Algorithm 5.2.38.) 


4. Nonzero locations: 


3000: | + | 0000] 00 | 18 | 35 3021: | + | 0000] 00] 01] 05 


3001: + 2051 00 | 05 | 09 3022: + 0000 | 04 | 12 | 31 
3002: + 2050 | 00 | 05 | 10 3023: + 0001 | 00 | 01 | 52 
3003: + 0001 | 00 | 00 | 49 3024: + 0050 | 00 | 01 | 53 
3004: + 0499 | 01 | 05 | 26 3025: + 3020 | 00 | 02 | 45 
3005: + 3016 00 | 01 | 41 3026: + 0000 | 04 | 18 | 37 
3006: + 0002 00 | 00 | 50 3027: + 0024 | 04 | 05 | 12 
3007: + 0002 00 | 02 | 51 3028: + 3019 | 00 | 00 | 45 
3008: + 0000 00 | 02 | 48 3029: + 0000 | 00 | 02 | 05 
3009: + 0000 02 | 02 | 55 0000: + 2 
3010: = 0001 03 | 05 | 04 1995: + | 06 | 09 | 19 | 22 | 23 
3011: + 3006 00 | 01 | 47 1996: + | 00 | 06 | 09 | 25 | 05 
3012: = 0001 | 03 | 05 | 56 1997: + | 00 | 08 | 24) 15 | 04 
3013: + 0001 | 00 | 00 | 51 1998: + | 19 | 05 | 04] 00 | 17 
3014: + 3008 | 00 | 06 | 39 1999: + |19|09/ 14) 05 | 22 
3015: + 3003 | 00 | 00 | 39 2024: + 2035 
3016: + 1995 00 | 18 | 37 2049: + 2010 
3017: + 2035 00 | 02 | 52 2050: + | 3 
3018: = 0050 00 | 02 | 53 2051: = 499 
3019: ji oe 90 09193 (the latter two may be interchanged, with 
3020: E 0001 05 | 05 | 08 corresponding changes to 3001 and 3002) 


5. Each OUT waits for the previous printer operation to finish (from the other buffer). 


6. (a) If n is not prime, by definition n has a divisor d with 1 < d < n. If d > yn, 
then n/d is a divisor with 1 < n/d < yn. (b) If N is not prime, N has a prime 
divisor d with 1 < d < vN. The algorithm has verified that N has no prime divisors < 
p = PRIME[K]; also N = pQ +R < pQ +p < p? +p < (p+1)°. Any prime divisor of N is 
therefore greater than p +1 > vN. 
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We must also prove that there will be a sufficiently large prime less than N when N 
is prime, namely that the (k + 1)st prime pz+1 is less than pe + pr; otherwise K would 
exceed J and PRIME[K] would be zero when we needed it to be large. The necessary 
proof follows from “Bertrand’s postulate”: If p is prime there is a larger prime less 


than 2p. 


7. (a) It refers to the location of line 29. (b) The program would then fail; line 14 
would refer to line 15 instead of line 25; line 24 would refer to line 15 instead of line 12. 


8. It prints 100 lines. If the 12000 characters on these lines were arranged end to 
end, they would reach quite far and would consist of five blanks followed by five A’s 
followed by ten blanks followed by five A’s followed by fifteen blanks ... followed by 
5k blanks followed by five A’s followed by 5(k + 1) blanks ... until 12000 characters 
have been printed. The third-from-last line ends with AAAAA and 35 blanks; the final 
two lines are entirely blank. The total effect is one of OP art. 


9. The (4:4) field of each entry in the following table holds the maximum F setting; 
the (1:2) field is the location of an appropriate validity-check routine. 


B EQU 1(4:4) BEGIN LDA INST 
BMAX EQU B-1 CMPA VALID(3:3) 
UMAX EQU 20 JG BAD I field > 6? 
TABLE NOP GOOD (BMAX) LD1 INST(5:5) 
ADD FLOAT(5:5) DEC1 64 
SUB FLOAT(5:5) JiNN BAD C field > 64? 
MUL FLOAT(5:5) CMPA TABLE+64,1(4:4) 
DIV FLOAT(5:5) JG BAD F field > F max? 
HLT GOOD LD1 TABLE+64,1(1:2) Jump to special 
SRC GOOD JMP 0,1 routine. 
MOVE MEMORY (BMAX) FLOAT CMPA VALID(4:4) F = 6 allowed on 
LDA FIELD(5:5) JE MEMORY arithmetic op 
a FIELD ENTA O 
STZ FIELD(5:5) LDX INST(4:4) This is a tricky 
JBUS MEMORY (UMAX) DIV =9= way to check 
IOC GOOD(UMAX) STX *+1(0:2) for a valid 
IN MEMORY (UMAX) INCA O partial field. 
OUT MEMORY (UMAX) DECA 5 
JRED MEMORY (UMAX) JAP BAD 
JLE MEMORY MEMORY LDX INST(3:3) 
JANP MEMORY JXNZ GOOD IFI = 0, 
see LDX INST(0:2) ensure the 
JXNP MEMORY JXN BAD address is a 
ENNA GOOD DECX 3999 valid memory 
tee JXNP GOOD location. 
ENNX GOOD JMP BAD 
CMPA FLOAT(5:5) VALID CMPX 3999,6(6) I 
CMP1 FIELD(5:5) 
CMPX FIELD(5:5) 


10. The catch to this problem is that there may be several places in a row or column 
where the minimum or maximum occurs, and each is a potential saddle point. 
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Solution 1: In this solution we run through each row in turn, making a list of all columns 
in which the row minimum occurs and then checking each column on the list to see if 
the row minimum is also a column maximum. rX = current min; rI1 traces through the 
matrix, going from 72 down to zero unless a saddle point is found; rl2 = column index 
of rl1; rI3 = size of list of minima. Notice that in all cases the terminating condition 
for a loop is that an index register is < 0. 


* SOLUTION 1 


A10 EQU 1008 Location of aio 
LIST EQU 1000 
START ENT1 9*8 Begin at the lower right corner. 
ROWMIN ENT2 8 Now rll is at column 8 of its row. 
2H LDX A10,1 Candidate for row minimum 
ENT3 0 List empty 
4H INC3 1 
ST2 LIST,3 Put column index in list. 
1H DEC1 1 Go left one. 
DEC2 1 
J2Z COLMAX Done with row? 
3H CMPX 410,1 
JL 1B Is rX still minimum? 
JG 2B New minimum? 
JMP 4B Remember another minimum. 
COLMAX LD2 LIST,3 Get column from list. 
INC2 9*8-8 
1H CMPX 410,2 
JL NO Is row min < column element? 
DEC2 8 
J2P 1B Done with column? 
YES INC1 A10+8,2 Yes; rll + address of saddle. 
HLT 
NO DEC3 1 Is list empty? 


J3P COLMAX No; try again. 
J1P ROWMIN Have all rows been tried? 
HLT Yes; rll = 0, no saddle. Į 


Solution 2: An infusion of mathematics gives a different algorithm. 


Theorem. Let R(i) = min; aij, C(j) = maxi aij. The element aj, j. is a saddle point 
if and only if R(io) = max; R(i) = C(jo) = min; C (å). 


Proof. If aiojo is a saddle point, then for any fixed i, R(io) = C(jo) > aijo > R(i); so 
R(io) = max; R(i). Similarly C(jo) = min; C(j). Conversely, we have R(t) < aij < 
C(j) for all i and j; hence R(io) = C (jo) implies that aigjo is a saddle point. J 


(This proof shows that we always have max; R(i) < min; C(j). So there is no 
saddle point if and only if all the R’s are less than all the C’s.) 

According to the theorem, it suffices to find the smallest column maximum, then 
to search for an equal row minimum. During Phase 1, rll = column index; rI2 runs 
through the matrix. During Phase 2, rll = possible answer; rl2 runs through the 
matrix; r13 = row index times 8; rl4 = column index. 
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* SOLUTION 2 
CMAX EQU 1000 
A10 EQU CMAX+8 


PHASE1 ENT1 8 Start at column 8. 
3H ENT2 9*8-8,1 Start at row 9. 
JMP 2F 
1H CMPX 410,2 Is rX still maximum? 
JGE *+2 
2H LDX 410,2 New maximum in column 
DEC2 8 
J2P 1B 
STX CMAX+8,2 Store column maximum. 
J2Z 1F First time? 
CMPA CMAX+8,2 rA still min max? 
JLE ¥*+2 
1H LDA CMAX+8,2 
DEC1 1 Move left a column. 
JIP 3B 
PHASE2 ENT3 9*8 At this point rA = min; C(J) 
3H ENT2 0,3 Prepare to search a row. 
ENT4 8 
1H CMPA A10,2 Is min; C(j) > afi, j]? 
JG NO No saddle in this row 
JL 2F 
CMPA CMAX,4 Is afi, j] = C(j)? 
JNE 2F 
ENT1 A10,2 Remember a possible saddle point. 
2H DEC4 1 Move left in row. 
DEC2 1 
J4P 1B 
HLT A saddle point was found. 
NO DEC3 8 
J3P 3B Try another row. 
ENT1i 0 
HLT rll = 0; no saddle. J 


We leave it to the reader to invent a still better solution in which Phase 1 records 
all possible rows that are candidates for the row search in Phase 2. It is not necessary 
to search all rows, just those io for which C(jo) = min; C(j) implies aigjo = C(Jo). 
Usually there is at most one such row. 

In some trial runs with elements selected at random from {0, 1,2,3,4}, solution 1 
required approximately 730u to run, while solution 2 took about 530u. Given a matrix 
of all zeros, solution 1 found a saddle point in 137u, solution 2 in 524u. 

If an m x n matrix has distinct elements, and m > n, we can solve the problem 
by looking at only O(m + n) of them and doing O(mlogn) auxiliary operations. See 
Bienstock, Chung, Fredman, Schäffer, Shor, and Suri, AMM 98 (1991), 418-419. 


11. Assume an m x n matrix. (a) By the theorem in the answer to exercise 10, all 
saddle points of a matrix have the same value, so (under our assumption of distinct 
elements) there is at most one saddle point. By symmetry the desired probability is 
mn times the probability that ai: is a saddle point. This latter is 1/(mmn)! times the 
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number of permutations with a12 > a11, .--, Qin > G11, Q11 > G21, ---, Q11 > Ami; this 
is 1/(m + n — 1)! times the number of permutations of m + n — 1 elements in which 
the first is greater than the next (m — 1) and less than the remaining (n — 1), namely 
(m —1)!(n —1)!. The answer is therefore 


mn(m—1)!(n-1)!/(m+n—1)!= (m4 a 


n 


In our case this is 17/ ea only one chance in 1430. (b) Under the second assumption, 
an entirely different method must be used. The probability equals the probability that 
there is a saddle point with value zero plus the probability that there is a saddle point 
with value one. The former is the probability that there is at least one column of 
zeros; the latter is the probability that there is at least one row of ones. The answer 
is (1 — (1 — 27™)”) + (1 — (1 — 27”)™); in our case it comes to 924744796234036231/ 
18446744073709551616, about 1 in 19.9. An approximate answer is n2~™ + m27”. 
12. M. Hofri and P. Jacquet [Algorithmica 22 (1998), 516-528] have analyzed the case 
when the m x n matrix entries are distinct and in random order. The running times of 
the two MIX programs are then respectively (6mn+5mH,+8m+6+5(m+1)/(n—-1))u4 
O((m+n)?/(™2")) and (5mn + 2nHm +7m+7n+9Hn)u+ O(1/n) + O((log n)?/m), 
as m — oo and n — oo, assuming that (logn)/m —> 0. 

13. * CRYPTANALYST PROBLEM (CLASSIFIED) 


TAPE EQU 20 Input unit number 
TYPE EQU 19 Output unit number 
SIZE EQU 14 Input block size 
OSIZE EQU 14 Output block size 
TABLE EQU 1000 Table of counts 
ORIG TABLE (initially zero 
CON -1 except entries for 
ORIG TABLE+46 blank space and 
CON -1 asterisk) 
ORIG 2000 
BUF1 ORIG *+SIZE First buffer area 
CON -1 “Sentinel” at end of buffer 
CON *+1 Reference to second buffer 
BUF2 ORIG *+SIZE Second buffer 
CON -1 “Sentinel” 
CON BUF1 Reference to first buffer 
BEGIN IN BUF1(TAPE) Input first block. 
ENT6 BUF2 


1H IN 0,6(TAPE) Input next block. 
LD6 SIZE+1,6 During this input, prepare 


ENTS 0,6 to process the previous one. 
JMP 4F 
2H INCA 1 Han 
STA TABLE,1 Update table entry. isa 
3H SLAX 1 Ps 
should 
STA ¥*+1(2:2) rll + next char. 
run as 
ENT1 O 
fast as 
LDA TABLE,1 


ibl 
JANN 2B Normal character? Deen 
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JiNZ 
JXP 
INC5 
4H LDX 
JXNN 
JMP 
3H ENT1 
2H LDA 
JANP 
CHAR 
JBUS 
ST1 
STA 
STX 
OUT 
1H CMP1 
INC1 
JL 
HLT 
ANS ALF 
ALF 
CHAR ALF 
FREQ ALF 
ORIG 
END 


TABLE, 1 
1F 


* (TYPE) 
CHAR(1:1) 
CHAR(4:5) 
FREQ 

ANS (TYPE) 
=63= 

1 

2B 


C NN 
NNNNN 
ANS+0SIZE 
BEGIN 


Asterisk? 
Skip over a blank. 


rX + five chars. 

Jump if not a sentinel. 

Done with block. 

Begin the endgame: rll + ‘A’. 


Skip zero answers. 


Convert to decimal. 
Wait till the typewriter is ready. 


Type one answer. 


Up to 63 character 
codes are counted 


The output buffer 


Rest of buffer is blank 
The literal constant =63= comes here. J 


For this problem, buffering of output is not desirable since it could save at most 7u of 
time per line output. 


14. To make the problem more challenging, the following solution due in part to 
J. Petolino uses a lot of trickery in order to reduce execution time. Can the reader 
squeeze out any more microseconds? 


* DATE OF EASTER 
EASTER STJ EASTX 
STX Y 
ENTA 0 
DIV =19= 
STX GMINUS1(0 
LDA Y 
MUL =1//100+1 
INCA 61 
STA CPLUS60(1 
MUL =3//4+1= 
STA XPLUS57(1 
CPLUS60 ENTA * 
MUL =8//25+1= 


GMINUS1 ENT2 * 
ENT1 1,2 
INC2 1,1 
INC2 0,2 


:2) 
= (see 
:2) 
:2) 
rÀ & Z + 24. 


E5. 
ril + G. 


1.3.2 


XPLUS57 


3H 
2H 
4H 


20MINUSN 


1H 


2H 


EASTX 


MARCH 
APRIL 
ANS 
DAY 
MONTH 


YEAR 


INC2 
INC2 
INC2 
INCA 
SRAX 
DIV 
DECX 
JXN 
DECX 
JXP 
JXN 
DEC1 
J1NP 
INCX 
DECX 
STX 
LDA 
MUL 
ADD 
SUB 
ENN1 
INCA 
SRAX 
DIV 
SLAX 
DECA 
JAN 
DECA 
CHAR 
LDA 
JMP 
CHAR 
LDA 
JBUS 
STA 
STX 
LDA 
CHAR 
STX 
OUT 
JMP 


ALF 
ALF 
ALF 
ALF 
ALF 
ALF 
ALF 
ORIG 
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773,1 rI2 + 11G + 773. 


-*,2 rA + 11G + Z — X + 20 + 24- 30 (>0). 


=30= rX & E. 


29 E6. 

20MINUSN (0:2) 

Y E4. 
=1//4+1= 

Y 

XPLUS57 (1:2) rA + D-— 47. 


67,1 E7. 
5 Xe D+N 


-4,1 rå + 31- N 
1F E8. 


MARCH 


APRIL 
*(18) 
MONTH 
DAY (1:2) 
Y 


YEAR 

ANS (18) Print 
* 

MARCH 

APRIL 


DD 
MMMMM 


YYYYY 
*+20 
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BEGIN ENTX 1950 “driver” 
ENT6 1950-2000 routine, 
JMP EASTER uses the 
INC6 1 subroutine 
ENTX 2000,6 above. 
J6NP EASTER+1 
HLT 
END BEGIN I 


A rigorous justification for the change from division to multiplication in several places 
can be based on the fact that the number in rA is not too large. The program works 
with all byte sizes. 

[To calculate Easter in years < 1582, see CACM 5 (1962), 209-210. The first 
systematic algorithm for calculating the date of Easter was the canon paschalis due to 
Victorius of Aquitania (A.D. 457). There are many indications that the sole nontrivial 
application of arithmetic in Europe during the Middle Ages was the calculation of 
Easter date, hence such algorithms are historically significant. See Puzzles and Para- 
doxes by T. H. O’Beirne (London: Oxford University Press, 1965), Chapter 10, for 
further commentary; and see the book Calendrical Calculations by E. M. Reingold and 
N. Dershowitz (Cambridge Univ. Press, 2001) for date-oriented algorithms of all kinds.] 


15. The first such year is A.D. 10317, although the error almost leads to failure in 
A.D. 10108 + 19k for 0 < k < 10. 

Incidentally, T. H. O’Beirne pointed out that the date of Easter repeats with 
a period of exactly 5,700,000 years. Calculations by Robert Hill show that the most 
common date is April 19 (220400 times per period), while the earliest and least common 
is March 22 (27550 times); the latest, and next-to-least common, is April 25 (42000 
times). Hill found a nice explanation for the curious fact that the number of times any 
particular day occurs in the period is always a multiple of 25. 


16. Work with scaled numbers, Rn = 10°rn. Then Rn(1/m) = R if and only if 
10"/(R+ 4) < m < 10”/(R— 4); thus we find m, = |2-10"/(2R — 1). 


* SUM OF HARMONIC SERIES 
BUF ORIG *+24 
START ENT2 0 
ENT1 3 5-n 
ENTA 20 
OUTER MUL =10= 
STX CONST 2-10” 
DIV =2= 
ENTX 2 
JMP iF 
INNER STA R 
ADD R 
DECA 1 
STA TEMP 2R-—1 
LDX CONST 
ENTA 0 
DIV TEMP 
INCA 1 
STA TEMP mnp +1 
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1H STA S Partial sum 
STX M mM = Me 


ADD M Compute R = Rn(1/m) = 
SRAX 5 | (2 - 10” + m)/(2m)]. 


JAP INNER R>0? 
LDA S 10” Sn 


SLAX 0,1 Neat formatting 


INCA 40 Decimal point 


LDA CONST 
JiNN OUTER 
OUT BUF(18) 


END START I 


The output is 
0006.16 0008.449 0010.7509 0013. 05363 


in 65595u plus output time. (It would be faster to calculate Rn(1/m) directly when 
m < 10”/?V2, and then to apply the suggested procedure.) 


17. Let N = |[2-10"/(2m+1)|. Then Sn = Hn +O(N/10”)+5 g ([2-10"/(2k-1)|- 
[2-10"/(2k + 1)|)k/10” = Hy + O(m™) + O(m/10") — 1 + 2Həm — Hm = nin 104 
2y — 1 + 21n 2 + O(107”/?) if we sum by parts and set m ~ 10"/?. 

Incidentally, the next several values are Sg = 15.356262, S7 = 17.6588276, Sg = 
19.96140690, So = 22.263991779, and S10 = 24.5665766353; our approximation to Sio 
is © 24.566576621, which is closer than predicted. 


18. FAREY STJ OF Assume that rll contains n, where n > 1. 
STZ X zo + 0. 
ENTX 1 
STX Y yo < 1. 


STX X+1 zı l1. 
ST1 Y+1 yen. 
ENT2 0 k +0. 
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1H LDX Y,2 
INCX 0,1 


STA TEMP  |(yk +n)/yk+ı] 


STA Y+2,2 Yk+2 


STA X+2,2 £k42 
CMPA Y+2,2 Test if £k42 < Yk+2. 


INC2 1 kek+1. 
JL 1B If so, continue. 
9H JMP * Exit from subroutine. [| 


19. (a) Induction. (b) Let k > 0 and X = azk+ı — £k, Y = ayk+ı — yr, where 
a = |(ye + n)/yk+ı]. By part (a) and the fact that 0 < Y < n, we have X L Y and 
X/Y > £tk41/Yk+1. So if X/Y Æ cp+2/yR+2 we have, by definition, X/Y > £k42/Yk+2- 
But this implies that 


1 = XYk+1 — Y k41 = X tky 
YYk+ı YYk+ı Yo Yr+1 
= (ž “2 N (== aa) 
Y  yYyk42/  \Yk+2 Yeti 
1 i 1 2 Yk+1 + Y > n > 1 


© YYkr42  Yk+1Yk+42 YYk+1yk42  YYk+1yk+2  Yypsi- 


Historical notes: C. Haros gave a (more complicated) rule for constructing such 
sequences, in J. de l’École Polytechnique 4, 11 (1802), 364-368; his method was correct, 
but his proof was inadequate. Several years later, the geologist John Farey indepen- 
dently conjectured that £k/yp is always equal to (£k—-1 + £k+1)/(Yk-1 + yk+1) [Philos. 
Magazine and Journal 47 (1816), 385-386]; a proof was supplied shortly afterwards by 
A. Cauchy [Bull. Société Philomathique de Paris (3) 3 (1816), 133-135], who attached 
Farey’s name to the series. For more of its interesting properties, see G. H. Hardy and 
E. M. Wright, An Introduction to the Theory of Numbers, Chapter 3. 


20. * TRAFFIC SIGNAL PROBLEM 


BSIZE EQU 1(4:4) Bytesize 
2BSIZE EQU 2(4:4) Twice bytesize 
DELAY STJ 1F If rA contains n, 
DECA 6 this subroutine 
DECA 2 waits max(n, 7) u 
JAP *-1 exactly, not including 
JAN *+2 the jump to the subroutine 
NOP 


1H JMP * 
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22. 


FLASH 


1H 


1H 


2H 
WAIT 
TRIP 


BEGIN 


AGREEN 
AAMBER 
BGREEN 
BAMBER 


* JOSEP. 


STJ 2F 

ENT2 8 

LDA =49991= 
JMP DELAY 
DECX 0,1 

LDA =49996= 
JMP DELAY 
INCX 0,1 

DEC2 1 

J2Z 1F 

LDA * 

JMP 1B 

LDA =399992= 
JMP DELAY 
JMP * 

JNOV * 

INCX BSIZE 
ENT1 2BSIZE 
JMP FLASH 
LDX BAMBER 
LDA =799995= 
JMP DELAY 
LDX AGREEN 
LDA =799996= 
JMP DELAY 
INCX 1 

ENT1 2 

JMP FLASH 
LDX AAMBER 
JOV *+1 

LDA =499994= 
JMP DELAY 
LDX BGREEN 
LDA =1799994= 
JMP DELAY 
JMP WAIT 

ALF CABA 
ALF  CBBB 
ALF ACAB 
ALF BCBB 
END BEGIN 


HUS PROBLEM 


N EQU 24 
M EQU 11 


X ORI 
OH ENT 
STZ 
ST1 
DEC 
J1P 


G *+N 
1 N-1 


11 


PWNDPRWRrRONRFBRWNOWNOAONFRFOTIAGAWAAHRTIBNYFP BWM OAN TK 
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This subroutine flashes the 
appropriate DON’T WALK light 


Turn light off. 


“DON?T WALK” 


Repeat eight times. 
Waste 2u of time. 

Get back in synch. 

Set amber 2u after exit. 


Del Mar green until tripped 
DON’T WALK on Del Mar 


Flash Del Mar. 
Amber on boulevard 


Wait 8 seconds. 
Green for avenue 


Wait 8 seconds. 
DON’T WALK on Berkeley 


Flash Berkeley. 
Amber on avenue 
Cancel redundant trip. 


Wait 5 seconds. 
Green on boulevard 


Wait at least 18 
seconds. 

Green for avenue 

Amber for avenue 

Green for boulevard 

Amber for boulevard 


1 Set each cell to the 

1 number of the next man 
N-1 in the sequence. 
N-1 
N-1 
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ENTA 1 1 (Now rll = 0) 
1H ENT2 M-2 N-1 (Assume M > 2) 
LD1 X,1 (M — 2)(N—1) Count around 
DEC2 1 (M — 2)(N —1) the circle. 
J2P *-2 (M — 2)(N —1) 
LD2 X,1 N-1 rll = lucky man 
LD3 X,2 N-1 rl2 = doomed man 
CHAR N-1 rl3 = next man 
STX X,2(4:5) N-1 Store execution number. 
NUM N-1 
INCA 1 N-1 
ST3 X,1 N-1 Take man from circle. 
ENT1 0,3 N-1 
CMPA =N= N-1 
JL 1B N-1 
CHAR 1 One man left; 
STX X,1(4:5) 1 he is clobbered too. 
OUT X(18) 1 Print the answer. 
HLT 1 
END OB l 


The last man is in position 15. The total time before output is (4(N—1)(M+7.5)+16)u. 
Several improvements are possible, such as D. Ingalls’s suggestion to have three-word 
packets of code ‘DEC2 1; J2P NEXT; JMP OUT’, where OUT modifies the NEXT field so as to 
delete a packet. An asymptotically faster method appears in exercise 5.1.1—5. 


SECTION 1.3.3 
1. (12 4)(365). 


2. a1 c, c+ f; bad. The generalization to arbitrary permutations is clear. 


3 abcdef 
“\dbfecaey 

4. (adcfe). 

5. 12. (See exercise 20.) 


6. The total time decreases by 8u for every blank word following a “(”, because lines 
30-32 cost 4u while lines 26-28, 33-34, 36-38 cost 12u. It decreases by 2u for every 
blank word following a name, because lines 68-71 cost 5u while 42—46 or 75-79 cost Tu. 
Initial blanks and blanks between cycles do not affect the execution time. The position 
of blanks has no effect whatever on Program B. 


7 X=2, Y =29,M=5,N=7,U =3, V = 1. Total, by Eq. (18), 2161u. 


8. Yes; we would then keep the inverse of the permutation, so that x; goes to x; if 
and only if T[j] = i. (The final cycle form would then be constructed from right to 
left, using the T table.) 


9. No. For example, given (6) as input, Program A will produce ‘(ADG) (CEB)’ as 
output, while Program B produces ‘(CEB) (DGA)’. The answers are equivalent but not 
identical, due to the nonuniqueness of cycle notation. The first element chosen for a 
cycle is the leftmost available name, in the case of Program A, and the last available 
distinct name to be encountered from right to left, in Program B. 
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10. (1) Kirchhoff’s law yields A = 1 + C — D; B= A+J+P-1;C=B-(P- L); 
E = D — L; G = E; Q = Z; W = S. (2) Interpretations: B = number of words of 
input = 16X — 1; C = number of nonblank words = Y; D = C — M; E = D — M; 
F = number of comparisons in names table search; H = N; K = M; Q = N; R= U; 
S= R- V; T =N — V since each of the other names gets tagged. (3) Summing up, 
we have (4F + 16Y +80X +21N — 19M +9U — 16V )u, which is somewhat better than 
Program A because F is certainly less than 16NX. The time in the stated case is 983u, 
since F = 74. 


11. “Reflect” it. For example, the inverse of (ac f)(b d) is (db)(fca). 


12. (a) The value in cell L+mn-—1 is fixed by the transposition, so we may omit it from 
consideration. Otherwise if x = n(i — 1) + (j — 1) < mn — 1, the value in L + æ should 
go to cell L+ maz mod N = L+(mn(i—1)+m(j—1)) mod N = L+m(j—1)+(i—1), 
since mn = 1 (modulo N) and 0 < m(j — 1) + (i — 1) < N. (b) If one bit in each 
memory cell is available (for example, the sign), we can “tag” elements as we move 
them, using an algorithm like Algorithm I. [See M. F. Berman, JACM 5 (1958), 383- 
384.] If there is no room for a tag bit, tag bits can be kept in an auxiliary table, or 
else a list of representatives of all non-singleton cycles can be used: For each divisor 
d of N, we can transpose those elements that are multiples of d separately, since m is 
prime to N. The length of the cycle containing x, when gced(x, N) = d, is the smallest 
integer r > 0 such that m” = 1 (modulo N/d). For each d, we want to find y(N/d)/r 
representatives, one from each of these cycles. Some number-theoretic methods are 
available for this purpose, but they are not simple enough to be really satisfactory. 
An efficient but rather complicated algorithm can be obtained by combining number 
theory with a small table of tag bits. [See N. Brenner, CACM 16 (1973), 692—694.] 
Finally, there is a method analogous to Algorithm J; it is slower, but needs no auxiliary 
memory, and it performs any desired permutation in situ. [See P. F. Windley, Comp. J. 
2 (1959), 47-48; D. E. Knuth, Proc. IFIP Congress (1971), 1, 19-27; E. G. Cate and 
D. W. Twigg, ACM Trans. Math. Software 3 (1977), 104-110; F. E. Fich, J. I. Munro, 
and P. V. Poblete, SICOMP 24 (1995), 266—278.] 


13. Show by induction that, at the beginning of step J2, X[i] = +j if and only if 
j > mand j goes to i under 7; X[i] = —j if and only if i goes to j under 7**!, where 
k is the smallest nonnegative integer such that 7” takes i into a number < m. 


14. Writing the inverse of the given permutation in canonical cycle form and dropping 
parentheses, the quantity A — N is the sum of the number of consecutive elements 
greater than a given element and immediately to its right. For example, if the original 
permutation is (1 6 5)(3 7 8 4), the canonical form of the inverse is (3 4 8 7)(2)(1 5 6); 
set up the array 


3 4 8 7 1 5 6 
e—eo o o oe—eo o 
e—eo o oe 

e o ° 


and the quantity A is the number of “dots,” 16. The number of dots below the kth 
element is the number of right-to-left minima in the first k elements (there are 3 dots 
below 7 in the example above, since there are 3 right-to-left minima in 3487). Hence 
the average is Hı + H2+---+ Hn = (n + 1)Hn - n. 


15. If the first character of the linear representation is 1, the last character of the 
canonical representation is 1. If the first character of the linear representation is m > 1, 
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then “...1m...” appears in the canonical representation. So the only solution is the 
permutation of a single object. (Well, there’s also the permutation of no objects.) 


16. 1324, 4231, 3214, 4213, 2143, 3412, 2413, 1243, 3421, 1324,.... 


17. (a) The probability pm that the cycle is an m-cycle is n!/m divided by n! Hn, so 
Pm = 1/(mHn). The average length is pı +2p2+3p3+-+- = 0" _, m/(mHn) = n/Hn. 
(b) Since the total number of m-cycles is n!/m, the total number of appearances of 
elements in m-cycles is n!. Each element appears as often as any other, by symmetry, 
so k appears n!/n times in m-cycles. In this case, therefore, pm = 1/n for all k and m; 
the average is )>”"_, m/n = (n+ 1)/2. 

18. See exercise 22(e). 

19. |Pro — n!/e| = n!/(n +1)! — n!/(n+2)!4+---, an alternating series of decreasing 
magnitudes, which is less than n!/(n +1)! < 3. 


20. There are a1 +a2+--- cycles in all, which can be permuted among one another, 
and each m-cycle can be independently written in m ways. So the answer is 


(ai + ag +--+ )E1EE 297 3% |, š 
21. 1/(ay!1*a2q!2™ ...) if n = ay + 2a2 + ---; zero otherwise. 


Proof. Write out ai 1-cycles, az 2-cycles, etc., in a row, with empty positions; for 
example if aj = 1, a2 = 2, a3 = a4 = --: = 0, we would have “(-) (--) (--)”. Fill 
the empty positions in all n! possible ways; we obtain each permutation of the desired 
form exactly ay! 1°tag! 2°? ... times. 

22. (a) Ifki+2k2+--- = n, the probability in (ii) is Į [~o f(w, J, kj), which is assumed 
to equal (1 — w) w"/(ki! 1"! k2! 2*2 ...); hence 


fw, m, kim +1) _ a) ck ae 
f(w,m, km) (Ise) If J kj T djm) m(km +1)" 


m 


Therefore by induction f(w,m,k) = (w™/m)* f(w,m,0)/k!, and condition (i) implies 
that f(w,m,k) = (w™/m)*e-’"/""/k!. [In other words, am is chosen with a Poisson 
distribution; see exercise 1.2.10-15.] 


wO E (Irw) =0-wu" F Poska.) = mt 
ky +2ko+--=n \j>0 ky +2ko+--=n 
ky ,h2 5.1.20 ky ,kg,...20 
Hence the probability that ai + 2a2+--»<nis (1—w)(ltwt---+w") =1- wt, 


(c) The average of ¢ is 


>( So Oki, ka, ---) Pr (a = kiraz = ho--)) 


n>0 \ ky+2kq+---=n 


= 0-0 Dw"/ 5 Als bay--) k11 1k12". 


n>0 ki +2koə+ =n 
(d) Let ¢(a1,a2,...) = a2 + a4 + as +---. The average value of the linear 
combination ¢ is the sum of the average values of a2, a4, &œ6,...; and the average value 
of Am is 


2 RF (wim, k) =), = (yer = 


k>0 k>1 
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Therefore the average value of ¢ is 


w w w _l-w 
2 4 2 


The desired answer is 4H [n/2|- 


(Hw? + Hw? + Hawt + How? + Hew? +---). 


(e) Set d(a1,a2,...) = z°™, and observe that the average value of ¢ is 


k 1 (w™z\* —w™/m _ i w™(z-1)/m _ wr z-1\3 
Drema = D A (EZE ee ween Ae (ez) 


! j! 
k>0 k>0 j>0 


-0-45 u"( 5 55) = (1 w) X` w”Gnm(2). 


j! 
O0<j<n/m n>0 


Hence 


_ L¢z-1y, __1 (=1/m)’ 
Gnm(z) = D = ( mi ) ; Pnkm = hel a a an 
0<j<n/m 0<j<n/m—k 

the statistics are (min 0, ave 1/m, max |n/m], dev \/1/m), when n > 2m. 
23. The constant À is [5° exp(—t — Ei (t)) dt, where E,(x) = f? e~‘dt/t. See Trans. 
Amer. Math. Soc. 121 (1966), 340-357, where many other results are proved, in par- 
ticular that the average length of the shortest cycle is approximately e~” Inn. Further 
terms of the asymptotic representation of l„ have been found by Xavier Gourdon [Ph.D. 
thesis, Ecole Polytechnique (Paris, 1996)]; the series begins 


An 4 LA Aen! | (5e 3 ( 1)")n~? | (e | a( 1)"4 sw” } au” te, 


where w = e?7*/3, William C. Mitchell has calculated a high-precision value of À = 


.62432 99885 43550 87099 29363 83100 83724 41796+ [Math. Comp. 22 (1968), 411- 
415]; no relation between à and classical mathematical constants is known. The same 
constant had, however, been computed in another context by Karl Dickman in Arkiv 
for Mat., Astron. och Fys. 22A,10 (1930), 1-14; the coincidence wasn’t noticed until 
many years later [Theor. Comp. Sci. 3 (1976), 373]. 


24. See D. E. Knuth, Proc. IFIP Congress (1971), 1, 19-27. 


25. One proof, by induction on N, is based on the fact that when the Nth element is 
a member of s of the sets it contributes exactly 


fo) = D + r= A- I? = dso 
to the sum. Another proof, by induction on M, is based on the fact that the number 
of elements that are in Sm but not in S41 U---U Sm-—1 is 


[Sm] — 5 |S} N Sm| + 5 |S; SkASm|—-:-. 


1<j<M 1<j<k<M 


26. Let No = N and let Ne = doy cj <...<5,<u Si NN Sip Then 


Nea (TEV Neg + (TEP) Noga = 


is the desired formula. It can be proved from the principle of inclusion and exclusion 
itself, or by using the method of exercise 25 together with the fact that 


(a) a ae) a ee 
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27. Let S; be the multiples of m; in the stated range and let N = amı ... m+. Then 
|S; O Sk| = N/mjmx, etc., so the answer is 


1 1 1 1 
NN San D De aw(i-2).. 4-2), 


m 
1<j<t i<j<k<t IOR 


This also solves exercise 1.2.4—30, if we let m1,...,mz+ be the primes dividing N. 
28. See I. N. Herstein and I. Kaplansky, Matters Mathematical (1974), §3.5. 


29. When passing over a man, assign him a new number (starting with n +1). Then 
the kth man executed is number 2k, and man number j for j > n was previously 
number (27) mod (2n + 1). Incidentally, the original number of the kth man executed 
is 2n+1—(2n+1 —2k)QUs2nr/2n+1—2k))] | [Armin Shams, Proc. Nat. Computer Conf. 
2002, English papers section, 2 (Mashhad, Iran: Ferdowsi University, 2002), 29-33.] 


31. See CMath, Section 3.3. Let xo = jm and x41 = (m(x — n) — di) /(m—1), where 
1< di <m. Then a, = j if and only if apj = bkn+tp, where a, = m**1 (m 1)*, bk = 
m(m*—(m—1)*), and tk = a m*-!~*(m—1)'d;. Since ax L bp and the (m—1)* pos- 
sibilities for t are distinct, the average number of k-step fixed elements is (m—1)*/ax. 
32. (a) In fact, k—1 < mtp <k+2 when k is even; k — 2 < Tmk <k+1 when k is odd. 
(b) Choose the exponents from left to right, setting e, = 1 if and only if k and k+1 are 


in different cycles of the permutation so far. [Steven Alpern, J. Combinatorial Theory 
B25 (1978), 62—73.] 
33. For | = 0, let (a1, a02; B01, Bo2) = (T, p;€,€) and (a11, 212; B11, 612) = (€, €; T, P), 
where m = (14)(23), p = (15)(24), and e = (). 

Suppose we have made such a construction for some l > 0, where Op = Bon = () 
for0<j7<mand1<k<n. Then the permutations 


(Am5) ++ +1 Amt); Byim4i is Bmt) an)) = 
(0 Qj10,...,0 QjnO,T AjnT,..-,T QjnT, 
o Bjno,...,0 Byio,T BymT,.--,T By; 
a Byji0,...,0 Bjno, T By1T,---,T BynT, 
O AjnO,...,0 OQj10,T Aj!nT,-.-,T O/T) 


have the property that 


Aimi) Biim4g)1 «+» AGim4i)(4n) Bm’) an) = 
a (12345)o7r (12345)ro (54321)o7 (54321)r 


if i = j and i’ = 7’, otherwise the product is (). Choosing o = (23)(45) and 7 = (345) 
will make the product (12345) as desired, when im + i' = jm + j’. 

The construction that leads from l to l+ 1 is due to David A. Barrington [J. Comp. 
Syst. Sci. 38 (1989), 150-164], who proved a general theorem by which any Boolean 
function can be represented as a product of permutations of {1,2,3,4,5}. With a 
similar construction we can, for example, find sequences of permutations (aj1,...,Qjn} 
By1,---,8jn) such that 


12345), ifi< j; 
04125102852... Qinbjn = { : ) i> 7; 


for 0 < i,j < m = 2?" when n = 6t! — 4, 
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34. Let N=m-+n. If m L n there is only one cycle, because every element can be 
written in the form am mod N for some integer a. And in general if d = gcd(m,n), 
there are exactly d cycles Co, Ci, ..., Ca-1, where C} contains the elements {7,7 + d, 

..;j +N — d} in some order. To carry out the permutation, we can therefore proceed 
as follows for 0 < j < d (in parallel, if convenient): Set t < a; and k + j; then 
while (k + m) mod N # j, set £k <— L(K4m) moan and k + (k +m) mod N; finally 
set £k + t. In this algorithm the relation (k + m) mod N Æ j will hold if and only if 
(k +m) mod N > d, so we can use whichever test is more efficient. [W. Fletcher and 
R. Silver, CACM 9 (1966), 326.] 


35. Let M =l+m+n and N =1+2m-+n. The cycles for the desired rearrangement 
are obtained from the cycles of the permutation on {0,1,..., N — 1} that takes k to 
(k +1+m) mod N, by simply striking out all elements of each cycle that are > M. 
(Compare this behavior with the similar situation in exercise 29.) Proof: When the 
hinted interchange sets £k < xp and xp + Xp for some k with k’ = (k+1+m) mod N 
and k” = (k’+1+m)modN and k’ > M, we know that xp = xp; hence the 
rearrangement aSy —> ypa replaces £k by xp. 

It follows that there are exactly d = gcd(l + m,m + n) cycles, and we can use an 
algorithm similar to the one in the previous exercise. 

A slightly simpler way to reduce this problem to the special case in exercise 34 is 


also noteworthy, although it makes a few more references to memory: Suppose y = y'y” 


where |y”] = |a|. Then we can change aßy'y” to 7" By'a, and interchange 76 with 
7’. A similar approach works if |a| > |y|. [See J. L. Mohammed and C. S. Subi, 


J. Algorithms 8 (1987), 113-121,] 


37. The result is clear when n < 2. Otherwise we can find a,b < n such that m takes 
a to b. Then (na) (bn) = (aa)(b p) for (n — 1)-cycles (aa) and (bf) if and only 
if r = (naa)(bBn). [See A. Jacques, C. Lenormand, A. Lentin, and J.-F. Perrot, 
Comptes Rendus Acad. Sci. 266 (Paris, 1968), A446—A448.| 


SECTION 1.4.1 


1. Calling sequence: JMP MAXN; or, JMP MAX100 if n = 100. 
Entry conditions: For the MAXN entrance, rI3 = n; assume n > 1. 
Exit conditions: Same as in (4). 


2. MAX50 STJ EXIT 
ENT3 50 
JMP 2F 


3. Entry conditions: rll = n. 

Exit conditions: Ifm > 1, rA, rI2, and rI3 are as in (4), with rl2 maximal < n; 
otherwise rA = CONTENTS(X+n), rl2 = n, and rI} = n — 1; 
rJ = EXIT+1; CI is unchanged if n < 1; otherwise CI is greater, 
equal, or less, according as the maximum is greater than X[1], 
equal to X[1] with rI2 > 1, or equal to X[1] with rl2 = 1. 


(The analogous exercise for (9) would of course be somewhat more complicated.) 


4. SMAX1 ENT1 1 r= 1 
SMAX STJ EXIT general r 
JMP 2F continue as before 
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DEC3 0,1 decrease by r 
J3P 1B 
EXIT JMP * exit. 
Calling sequence: JMP SMAX; or, JMP SMAX1 if r= 1. 
Entry conditions: r13 = n, assumed positive; for the SMAX entrance, rll = r, assumed 
positive. 
Exit conditions: rA = maxo<,<n/r CONTENTS(X + n — kr) = CONTENTS (X + rI2); and 
rl3 = (n — 1)modr+1—r=-—((—n) mod r). 
5. Any other register can be used. For example, 
Calling sequence: ENTA *+2 
JMP MAX100 
Entry conditions: None. 
Exit conditions: Same as in (4). 
The code is like (1), but the first instruction becomes ‘MAX100 STA EXIT(0:2)’. 


6. (Solution by Joel Goldberg and Roger M. Aarons.) 


MOVE STJ 3F 
STA 4F Save rA and rI2. 
ST2 5F(0:2) 


LD2 3F(0:2) rl2 4 address of ‘NOP A,I(F)’. 
LDA 0,2(0:3) rA<‘A,I’. 

STA ¥*+2(0:3) 

LD2 5F(0:2) Restore rI2, because I might be 2. 


ENTA * rA + indexed address. 
LD2 3F(0:2) 

LD2N 0,2(4:4) rl2¢ —F. 

J2Z 1F 

DECA 0,2 

STA 2F(0:2) 

DEC1 0,2 rll + rll +F. 

ST1 6F(0:2) 


2H LDA *,2 
6H STA *,2 


INC2 1 Increase rI2 until it becomes zero. 
J2N 2B 

1H LDA 4F Restore rA and rI2. 

5H ENT2 * 

3H JMP * Exit to the NOP instruction. 


4H CON O J 


7. (1) An operating system can allocate high-speed memory more efficiently if pro- 
gram blocks are known to be “read-only.” (2) An instruction cache in hardware will be 
faster and less expensive if instructions cannot change. (3) Same as (2), with “pipeline” 
in place of “cache.” If an instruction is modified after entering a pipeline, the pipeline 
needs to be flushed; the circuitry needed to check this condition is complex and time- 
consuming. (4) Self-modifying code cannot be used by more than one process at once. 
(5) Selfmodifying code can defeat a jump-trace routine (exercise 1.4.3.2-7), which is 
an important diagnostic tool for “profiling” (that is, for computing the number of times 
each instruction is executed). 


1.4.2 ANSWERS TO EXERCISES 529 


SECTION 1.4.2 


1. If one coroutine calls the other only once, it is nothing but a subroutine; so we 
need an application in which each coroutine calls the other in at least two distinct 
places. Even then, it is often easy to set some sort of switch or to use some property 
of the data, so that upon entry to a fixed place within one coroutine it is possible to 
branch to one of two desired places; again, nothing more than a subroutine would be 
required. Coroutines become correspondingly more useful as the number of references 
between them grows larger. 


2. The first character found by IN would be lost. [We started OUT first because lines 
58-59 do the necessary initialization for IN. If we wanted to start IN first, we’d have 
to initialize OUT by saying ‘ENT4 -16’, and clearing the output buffer if it isn’t known 
to be blank. Then we could make line 62 jump first to line 39.] 


3. Almost true, since ‘CMPA =10=’ within IN is then the only comparison instruction 
of the program, and since the code for ‘.’ is 40. (!) But the comparison indicator isn’t 
initialized; and if the final period is preceded by a replication digit, it won’t be noticed. 
[Note: The most nitpickingly efficient program would probably remove lines 40, 44, and 
48, and would insert ‘CMPA PERIOD’ between lines 26 and 27, ‘CMPX PERIOD’ between 
lines 59 and 60. The state of the comparison indicator should then become part of the 
coroutine characteristics in the program documentation] 


4. Here are examples from three rather different computers of historic importance: 
(i) On the IBM 650, using SOAP assembly language, we would have the calling se- 
quences ‘LDD <A’ and ‘LDD BP’, and linkage ‘A STD BX AX’ and ‘B STD AX BX’ (with 
the two linkage instructions preferably in core). (ii) On the IBM 709, using common 
assembly languages, the calling sequences would be ‘TSX A,4’ and ‘TSX B,4’; the linkage 
instructions would be as follows: 


A SXA_ BX,4 B SXA AX,4 
AX AXT 1-A1,4 BX AXT 1-B1,4 
TRA 1,4 TRA 1,4 


(iii) On the CDC 1604, the calling sequences would be “return jump” (SLJ 4) to A or B, 
and the linkage would be, for example, 
A: SLJ Bi; ALS O 
B: SLJ A1; SLJ A 
in two consecutive 48-bit words. 
5. ‘STA HOLDAIN; LDA HOLDAOUT’ between OUT and OUTX, and ‘STA HOLDAOUT; LDA 
HOLDAIN’ between IN and INX. 
6. Within A write ‘JMP AB’ to activate B, ‘JMP AC’ to activate C. Locations BA, BC, CA, 
and CB would, similarly, be used within B and C. The linkage is: 


AB STJ AX BC STJ BX CA STJ CX 
BX JMP Bi CX JMP Cl AX JMP Atl 
CB STJ CX AC STJ AX BA STJ BX 

JMP BX JMP CX JMP AX 


[Note: With n coroutines, 2(n — 1)n cells would be required for this style of linkage. If 
n is large, a “centralized” routine for linkage could of course be used; a method with 
3n + 2 cells is not hard to invent. But in practice the faster method above requires 
just 2m cells, where m is the number of pairs (i,j) such that coroutine i jumps to 
coroutine j. When there are many coroutines each independently jumping to others, 
the sequence of control is usually under external influence, as discussed in Section 2.2.5.] 
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SECTION 1.4.3.1 


1. FCHECK is used only twice, both times immediately followed by a call on MEMORY. 
So it would be slightly more efficient to make FCHECK a special entrance to the MEMORY 
subroutine, and also to make it put —R in rI2. 


2. SHIFT J5N ADDRERROR 3. MOVE J3Z CYCLE 
DEC3 5 JMP MEMORY 
J3P FERROR SRAX 5 
LDA AREG LD1 I1REG 
LDX XREG LDA SIGN1 
LD1 1F,3(4:5) JAP ¥*+3 
ST1 2F(4:5) JiNZ MEMERROR 
J5Z CYCLE STZ SIGN1(0:0) 
2H SLA 1 CMP1 =BEGIN= 
DEC5 1 JGE MEMERROR 
J5P 2B STX 0,1 
JMP STOREAX LDA CLOCK 
SLA 1 INCA 2 
SRA 1 STA CLOCK 
SLAX 1 INC1 1 
SRAX 1 ST1 I1REG 
SLC 1 INC5 1 
1H SRC 1 J DEC3 1 
JMP MOVE J 


4. Just insert ‘IN 0(16)’ and ‘JBUS *(16)’ between lines 003 and 004. (Of course on 
another computer this would be considerably different since it would be necessary to 
convert to MIX character code.) 


5. Central control time is 34u, plus 15u if indexing is required; the GETV subroutine 
takes 52u, plus 5u if L 4 0; extra time to do the actual loading is 11u for LDA or LDX, 
13u for LDi, 21u for ENTA or ENTX, 23u for ENT (add 2u to the latter two times if 
M = 0). Summing up, we have a total time of 97u for LDA and 55u for ENTA, plus 15u 
for indexing, and plus 5u or 2u in certain other circumstances. It would seem that 
simulation in this case is causing roughly a 50:1 ratio in speeds. (Results of a test run 
that involved 178u of simulated time required 8422u of actual time, a 47:1 ratio.) 


7. Execution of IN or OUT sets a variable associated with the appropriate input device 
to the time when transmission is desired. The ‘CYCLE’ control routine interrogates these 
variables on each cycle, to see if CLOCK has exceeded either (or both) of them; if so, 
the transmission is carried out and the variable is set to oo. (When more than two 
I/O units must be handled in this way, there might be so many variables that it 
would be preferable to keep them in a sorted list using linked memory techniques; see 
Section 2.2.5.) We must be careful to complete the I/O when simulating HLT. 


8. False; rl6 can equal BEGIN, if we “fall through” from the location BEGIN — 1. But 
then a MEMERROR will occur, trying to STZ into TIME! On the other hand, we always do 
have 0 < rl6 < BEGIN, because of line 254. 


SECTION 1.4.3.2 
1. Change lines 48 and 49 to the following sequence: 
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XREG ORIG *+2 JMP -1,1 
LEAVE STX XREG 1H JMP *+1 
ST1 XREG+1 STA -1,1 
LD1 JREG(0:2) LD1 XREG+1 
LDA -1,1 LDX XREG 
LDX 1F LDA AREG 
STX -1,1 LEAVEX JSJ * 
The operator ‘JSJ’ here is, of course, particularly crucial. 
2. * TRACE ROUTINE STA BUF+1,1(4:5) 
ORIG *+99 ENTA 8 
BUF CON O JNOV iF 
EEE EE lines 02-04 ADD BIG 
ST1 I1REG 1H JL 1F 
wane Laoag lines 05-07 INCA 1 
PTR ENT1 -100 JE iF 
JBUS *(0) INCA 1 
STA BUF+1,1(0:2) 1H STA BUF+1,1(3:3) 
EEE PET EEE EE lines 08-11 INC1 10 
STA BUF+2,1 JiN 1F 
EEE EESE lines 12-13 OUT BUF-99(0) 
LDA AREG ENT1 -100 
STA BUF+3,1 1H STi PTR(0:2) 
LDA I1REG LD1 I1REG 
STA BUFt+4,1 2 2 2 eee eee lines 14-35 
ST2 BUF+5,1 ST1 I1REG 
ST3 BUF+6,1 eee eee eee lines 36-48 
ST4 BUF+7,1 LDi I1REG 
ST5 BUFt8,1 cece ee ee eee lines 49-50 
ST6 BUF+9,1 B4 EQU 1(1:1) 
STX BUF+10,1 BIG CON B4-8,B4-1(1:1) J 


LDA JREG(0:2) 
A supplementary routine that writes out the final buffer and rewinds tape 0 should 
be called after all tracing has been performed. 


3. Tape is faster; and the editing of this information into characters while tracing 
would consume far too much space. Furthermore the tape contents can be selectively 
printed. 


4. A true trace, as desired in exercise 6, would not be obtained, since restriction (a) 
mentioned in the text is violated. The first attempt to trace CYCLE would cause a loop 
back to tracing ENTER+1, because PREG is clobbered. 


6. Suggestion: Keep a table of values of each memory location within the trace area 
that has been changed by the outer program. 


7. The routine should scan the program until finding the first jump (or conditional 
jump) instruction; after modifying that instruction and the one following, it should 
restore registers and allow the program to execute all its instructions up to that point, 
in one burst. [This technique can fail if the program modifies its own jump instructions, 
or changes non-jumps into jumps. For practical purposes we can outlaw such practices, 
except for STJ, which we probably ought to handle separately anyway.] 
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SECTION 1.4.4 


1. (a) No; the input operation might not yet be complete. (b) No; the input operation 
might be going just a little faster than the MOVE. This proposal is much too risky. 


2. ENT1 2000 
JBUS *(6) 
MOVE 1000(50) 
MOVE 1050(50) 
OUT 2000(6) J 


3. WORDOUT STJ 1F DEC5 100 

STA 0,5 JMP 2B 
INC5 1 * BUFFER AREAS 

2H CMP5 BUFMAX OUTBUF1 ORIG *+100 

1H JNE * ENDBUF1 CON ¥*+101 (ENDBUF2) 
OUT -100,5(V) OUTBUF2 ORIG *+100 
LD5 0,5 ENDBUF2 CON ENDBUF1 
ST5 BUFMAX BUFMAX CON ENDBUF1 | 


At the beginning of the program, give the instruction ‘ENT5 OUTBUF1’. At the end of 
the program, say 


LDA BUFMAX INC5 1 
DECA 100,5 CMP5 BUFMAX 
JAZ *+6 JNE *-3 

STZ 0,5 OUT -100,5(V) 


4. If the calculation time exactly equals the I/O time (which is the most favorable 
situation), both the computer and the peripheral device running simultaneously will 
take half as long as if they ran separately. Formally, let C be the calculation time for 
the entire program, and let T be the total I/O time required; then the best possible 
running time with buffering is max(C, T), while the running time without buffering is 
C +T; and of course $(C + T) < max(C,T) < C +T. 

However, some devices have a “shutdown penalty” that causes an extra amount 
of time to be lost if too long an interval occurs between references to that unit; in such 


a case, better than 2:1 ratios are possible. (See, for example, exercise 19.) 


5. The best ratio is (n + 1):1. 


ee = of T eal 
ENT6 INBUF2+99 ENT6 INBUF1+99 


(possibly preceded by IOC 0(U) to rewind the tape just in case it is necessary). 


7. One way is to use coroutines: 


INBUF1 ORIG *+100 INC6 1 
INBUF2 ORIG *+100 J6N 2B 
1H LDA INBUF2+100,6 IN INBUF 1 (U) 
JMP MAIN ENN6 100 
INC6 1 JMP 1B 
J6N 1B WORDIN STJ MAINX 
WORDIN IN INBUF2(U) WORDINX JMP WORDIN1 
ENN6 100 MAIN STJ WORDINX 
2H LDA INBUF1+100,6 MAINX JMP * Jj 


JMP MAIN 
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Adding a few more instructions to take advantage of special cases will actually make 
this routine faster than (4). 


8. At the time shown in Fig. 23, the two red buffers have been filled with line images, 
and the one indicated by NEXTR is being printed. At the same time, the program is 
computing between RELEASE and ASSIGN. When the program ASSIGNs, the green buffer 
indicated by NEXTG becomes yellow; NEXTG moves clockwise and the program begins to 
fill the yellow buffer. When the output operation is complete, NEXTR moves clockwise, 
the buffer that has just been printed turns green, and the remaining red buffer begins 
to be printed. Finally, the program RELEASEs the yellow buffer and it too is ready for 
subsequent printing. 


9,10, 11. 
time | action (N = 1) action (N = 2) action (N = 4) 

0 | ASSIGN(BUF1) ASSIGN (BUF 1) ASSIGN (BUF 1) 
1000 | RELEASE, OUT BUF1 RELEASE, OUT BUF1 RELEASE, OUT BUF1 
2000 | ASSIGN (wait) ASSIGN (BUF2) ASSIGN (BUF2) 
3000 RELEASE RELEASE 
4000 ASSIGN (wait) ASSIGN (BUF3) 
5000 RELEASE 
6000 ASSIGN (BUF4) 
7000 RELEASE 
8000 ASSIGN (wait) 
8500 | BUF1 assigned, output stops | BUF1 assigned, OUT BUF2 | BUF1 assigned, OUT BUF2 
9500 | RELEASE, OUT BUF1 RELEASE 

10500 | ASSIGN (wait) ASSIGN (wait) 
15500 RELEASE 


and so on. Total time when N = 1 is 110000u; when N = 2 it is 89000u; when N = 3 
it is 81500u; and when N > 4 it is 76000u. 


12. Replace the last three lines of Program B by 


STA 2F 

LDA 3F 

CMPA 15,5(5:5) 

LDA 2F 

LD5 -1,5 

DEC6 1 

JNE 1B 

JMP COMPUTE 

JMP *-1 (or JMP COMPUTEX) 


2H CON 0 
3H ALF uudis l 
13. JRED CONTROL (U) 
J6NZ *-1 J 


14. If N = 1 the algorithm breaks down (possibly referring to the buffer while I/O is 
in progress); otherwise the construction will have the effect that there are two yellow 
buffers. This can be useful if the computational program wants to refer to two buffers at 
once, although it ties up buffer space. In general, the excess of ASSIGNs over RELEASEs 
should be nonnegative and not greater than N. 
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15. U EQU 0 OUT BUF2(V) 
V EQU 1 IN BUF1(U) 
BUF1 ORIG *+100 OUT BUF3(V) 
BUF2 ORIG *+100 DEC1 3 
BUF3 ORIG *+100 JiP 1B 
TAPECPY IN BUF1(U) JBUS *(U) 
ENT1 99 OUT BUF1i(V) 
1H IN BUF2(U) HLT 
OUT BUF1(V) END TAPECPY J 
IN BUF3(U) 


This is a special case of the algorithm indicated in Fig. 26. 


18. Partial solution: In the algorithms below, t is a variable that equals 0 when the 
I/O device is idle, and 1 when it is active. 


Algorithm A’ (ASSIGN, a normal state subroutine). 
This algorithm is unchanged from Algorithm 1.4.4A. 


Algorithm R’ (RELEASE, a normal state subroutine). 
R1’. Increase n by one. 


R2’. If t = 0, force an interrupt, going to step B3’ (using the INT operator). J 


Algorithm B’ (Buffer control routine, which processes interrupts). 
B1’. Restart the main program. 
B2’. If n = 0, set t + 0 and go to BI’. 
B3’. Set t + 1, and initiate I/O from the buffer area specified by NEXTR. 


B4’. Restart the main program; an “I/O complete” condition will interrupt it and 
lead to step B5’. 

B5’. Advance NEXTR to the next clockwise buffer. 

B6’. Decrease n by one, and go to step B2’. J 
19. If C < L we can have tẹ = (k — 1)L, ux = tk + T, and vk = uz, + C if and only if 
NL>T+C. If C > L the situation is more complex; we can have up = (k-1)C+T 
and vk = kC + T if and only if there are integers a1 < a2 < -+> < an such that 
tk = (k — 1)L + akP satisfies up — T > th > vg_n for N < k < n. An equivalent 
condition is that NC > bp for N < k < n, where bk = C +T + ((k—1)(C — L)) mod P. 
Let cr = max{bı+1,..., bn, 0}; then cı decreases as l increases, and the smallest value 
of N that keeps the process going steadily is the minimum / such that &/l < C. 
Since a < C +T +P and a < L+T + n(C — L), this value of N never exceeds 
[min{C +T + P,L+T +n(C — L)}/C]. [See A. Itai and Y. Raz, CACM 31 (1988), 
1338-1342.] 

In the stated example we have therefore (a) N = 1; (b) N = 2; (c) N = 3, 
cn = 2.5; (d) N = 35, cw = 51.5; (e) N = 51, cw = 101.5; (f€) N = 41, cw = 102; 
(g) N = 11, cn = 109.5; (h) N = 3, cw = 149.5; (i) N = 2, cw = 298.5. 


SECTION 2.1 
1. (a) SUIT(NEXT(TOP)) = SUIT(NEXT(242)) = SUIT(386) = 4. (b) A 


2. Whenever V is a link variable (else CONTENTS(V) makes no sense) whose value is 
not A. It is wise to avoid using LOC in contexts like this. 
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3. Set NEWCARD < TOP, and if TOP Æ A set TOP + NEXT(TOP). 


4. C1. Set X < LOC(TOP). (For convenience we make the reasonable assumption 
that TOP = NEXT(LOC(TOP)), namely that the value of TOP appears in the 
NEXT field of the location where it is stored. This assumption is compatible 
with program (5), and it saves us the bother of writing a special routine for 
the case of an empty pile.) 


C2. If NEXT(X) # A, set X + NEXT(X) and repeat this step. 
C8. Set NEXT(X) +4 NEWCARD, NEXT(NEWCARD) + A, TAG(NEWCARD) + 1. J 


5. D1. Set X + LOC(TOP), Y «+ TOP. (See step C1 above. By hypothesis, Y 4 A. 
Throughout the algorithm that follows, X trails one step behind Y in the sense 
that Y = NEXT(X).) 

D2. If NEXT(Y) Æ A, set X + Y, Y + NEXT(Y), and repeat this step. 
D3. (Now NEXT(Y) = A, so Y points to the bottom card; also X points to the 
next-to-last card.) Set NEXT(X) + A, NEWCARD< Y. J 
6. Notations (b) and (d). Not (a)! CARD is a node, not a link to a node. 
7. Sequence (a) gives NEXT(LOC(TOP)), which in this case is identical to the value of 
TOP; sequence (b) is correct. There is no need for confusion; consider the analogous 


example when X is a numeric variable: To bring X into register A, we write LDA X, not 
ENTA X, since the latter brings LOC(X) into the register. 


8. Let rA =N, rll = X. 


ENTA 0 BI.N€O. INCA 1 B3. N+N+1. 
LD1 TOP X< TOP. LD1 0,1(NEXT) X + NEXT (X). 
JiZ *+4 B2. Is X= A? J1NZ *-2 I 
9. Let rl2 = X. 
PRINTER EQU 18 Unit number for line printer 
TAG EQU 1:1 
NEXT EQU 4:5 Definition of fields 
NAME EQU 0:5 
PBUF ALF PILE Message printed in case 
ALF EMPTY pile is empty 
ORIG PBUF+24 
BEGIN LD2 TOP Set X < TOP. 
J2Z 2F Is the pile empty? 
1H LDA 0,2(TAG) rA + TAG(X). 
ENT1 PBUF Get ready for MOVE instruction. 
JBUS * (PRINTER) Wait until printer is ready. 
JAZ *+3 Is TAG = 0 (is card face up)? 
MOVE PAREN (3) No: Copy parentheses. 
JMP ¥*+2 
MOVE BLANKS (3) Yes: Copy blanks. 
LDA 1,2(NAME) rA + NAME(X). 
STA PBUF+1 
LD2 0,2(NEXT) Set X < NEXT(X). 
2H OUT PBUF(PRINTER) Print the line. 
J2NZ 1B If X Æ A, repeat the print loop. 


DONE HLT 
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PAREN ALF ¢ 
BLANKS ALF 
ALF ) 
ALF I 
SECTION 2.2.1 


1. Yes. (Consistently insert all items at one of the two ends.) 


2. To obtain 325641, do SSSXXSSXSXXX (in the notation of the following exercise). 
The order 154623 cannot be achieved, since 2 can precede 3 only if it is removed from 
the stack before 3 has been inserted. 


3. An admissible sequence is one in which the number of X’s never exceeds the number 
of S’s if we read from the left to the right. 

Two different admissible sequences must give a different result, since if the two 
sequences agree up to a point where one has S and the other has X, the latter sequence 
outputs a symbol that cannot possibly be output before the symbol just inserted by 
the S of the former sequence. 


4. This problem is equivalent to many other interesting problems, such as the enumer- 
ation of binary trees, the number of ways to insert parentheses into a formula, and the 
number of ways to divide a polygon into triangles, and it appeared as early as 1759 in 
notes by Euler and von Segner (see Section 2.3.4.6). 

The following elegant solution uses a “reflection principle” due to J. Aebly and 
D. Mirimanoff [L’Enseignement Math. 23 (1923), 185-189]: There are obviously 7") 
sequences of S’s and X’s that contain n of each. It remains to evaluate the number of 
inadmissible sequences (those that contain the right number of S’s and X’s but violate 
the other condition). In any inadmissible sequence, locate the first X for which the 
X’s outnumber the S’s. Then in the partial sequence leading up to and including this 
X, replace each X by S and each S by X. The result is a sequence with (n+ 1) S’s 
and (n — 1) X’s. Conversely for every sequence of the latter type we can reverse the 
process and find the inadmissible sequence of the former type that leads to it. For 
example, the sequence XXSXSSSXXSSS must have come from SSXSXXXXXSSS. This 
correspondence shows that the number of inadmissible sequences is (e): Hence an = 
(3) J Ge 

Using the same idea, we can solve the more general “ballot problem” of probability 
theory, which essentially is the enumeration of all partial admissible sequences with a 
given number of S’s and X’s. This problem was actually resolved as early as 1708 
by Abraham de Moivre, who showed that the number of sequences containing l A’s 
and m B’s, and containing at least one initial substring with n more A’s than B’s, is 
f(l,m,n) = (aia): In particular, an = ee — f(n,n,1) as above. (De Moivre 
stated this result without proof [Philos. Trans. 27 (1711), 262-263]; but it is clear from 
other passages in his paper that he knew how to prove it, since the formula is obviously 
true when l > m + n, and since his generating-function approach to similar problems 
yields the symmetry condition f(l, m,n) = f(m + n,l — n,n) by simple algebra.) For 
the later history of the ballot problem and some generalizations, see the comprehensive 
survey by D. E. Barton and C. L. Mallows, Annals of Math. Statistics 36 (1965), 
236-260; see also exercise 2.3.4.4—32 and Section 5.1.4. 

We present here a new method for solving the ballot problem with the use of 
double generating functions, since this method lends itself to the solution of more 
difficult problems such as the question in exercise 11. 
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Let gnm be the number of sequences of S’s and X’s of length n, in which the 
number of X’s never exceeds the number of S’s if we count from the left, and in which 
there are m more S’s than X’s in all. Then an = G(2n)o- Obviously gnm is zero unless 
m+n is even. We see easily that these numbers can be defined by the recurrence 
relations 


J(nt1)m = Jn(m-1) + Jn(m+1) M >0, n>0; gom = dom: 


Consider the double generating function G(z,z) = X, m Qnmxz™ z”, and let g(z) = 
G(0, z). The recurrence above is equivalent to the equation 


1 1 1 . zg(z)— x 
This equation unfortunately tells us nothing if we set x = 0, but we can proceed by 
factoring the denominator as z(1 — ri(z)x)(1 —r2(z)x) where 


rı(z) = = +V1—422), re(z)= s(t — V1- 422). 

(Note that rı + r2 = 1/z; rire = 1.) We now proceed heuristically; the problem is to 
find some value of g(z) such that G(x, z) as given by the formula above has an infinite 
power series expansion in x and z. The function r2(z) has a power series, and r2(0) = 0; 
moreover, for fixed z, the value x = r2(z) causes the denominator of G(x, z) to vanish. 
This suggests that we should choose g(z) so that the numerator also vanishes when 
xz = r2(z); in other words, we probably ought to take zg(z) = r2(z). With this choice, 
the equation for G(x, z) simplifies to 


G(a,z) = r2(z) = N (rale) re. 


2(1 — r2(z)x) ay 


This is a power series expansion that satisfies the original equation, so we must have 
found the right function g(z). 

The coefficients of g(z) are the solution to our problem. Actually we can go further 
and derive a simple form for all the coefficients of G(x, z): By the binomial theorem, 


nf OR 1 
_ 2k+1 
ra(z) = Diz ( k nese 


k>0 


Let w = 2? and ro(z) = zf(w). Then f(w) = 0,59 Ax(1,—2)w* in the notation of 
exercise 1.2.6-25; hence E 


fwy = Jo Ax(r, —2)w*. 


k>0 


We now have 
G(x,z) = oy Am(n +1,2) 2", 


n,m 


so the general solution is 


- (7 T 1 = ( 2n )={ 2n | 
IMME (n-m) m41  \n-m n=-m-1/’ 


2n+2\ 2m +2 2n +1 2n +1 
J(2n+1)(2m+1) = a.) m4 2 = ( ) = ( i 
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5. Ifj <k and pj < pk, we must have taken p; off the stack before px was put on; if 
pj > Pr, we must have left py on the stack until after p; was put on. Combining these 
two rules, the condition i < j < k and pj < pr < pi is impossible, since it means that 
pj must go off before py, and after pi, yet p; appears after pp. 

Conversely, the desired permutation can be obtained by using the following algo- 
rithm: “For j = 1, 2,..., n, input zero or more items (as many as necessary) until pj 
first appears in the stack; then output pj.” This algorithm can fail only if we reach a 
j for which p; is not at the top of the stack but it is covered by some element pp for 
k > j. Since the values on the stack are always monotone increasing, we have pj < px. 
And the element pp must have gotten there because it was less than p; for some i < j. 

P. V. Ramanan [SICOMP 13 (1984), 167-169] has shown how to characterize 
the permutations obtainable when m auxiliary storage locations can be used freely in 
addition to a stack. (This generalization of the problem is surprisingly difficult.) 


6. Only the trivial one, 12...n, by the nature of a queue. 


7. An input-restricted deque that first outputs n must simply put the values 1, 2, 
..., Nn on the deque in order as its first n operations. An output-restricted deque that 
first outputs n must put the values pi p2...pn on its deque as its first n operations. 
Therefore we find the unique answers (a) 4132, (b) 4213, (c) 4231. 


8. When n < 4, no; when n = 5, there are four (see exercise 13). 


9. By operating backwards, we can get the reverse of the inverse of the reverse of 
any input-restricted permutation with an output-restricted deque, and conversely. This 
rule sets up a one-to-one correspondence between the two sets of permutations. 


10. (i) There should be n X’s and n combined S’s and Q’s. (ii) The number of X’s 
must never exceed the combined number of S’s and Q’s, if we read from the left. (iii) 
Whenever the number of X’s equals the combined number of S’s and Q’s (reading from 
the left), the next character must be a Q. (iv) The two operations XQ must never be 
adjacent in this order. 

Clearly rules (i) and (ii) are necessary. The extra rules (iii) and (iv) are added to 
remove ambiguity, since S is the same as Q when the deque is empty, and since XQ can 
always be replaced by QX. Thus, any obtainable permutation corresponds to at least 
one admissible sequence. 

To show that two admissible sequences give different permutations, consider se- 
quences that are identical up to a point, and then one sequence has an S while the other 
has an X or Q. Since by (iii) the deque is not empty, clearly different permutations 
(relative to the order of the element inserted by S) are obtained by the two sequences. 
The remaining case is where sequences A, B agree up to a point and then sequence A 
has Q, sequence B has X. Sequence B may have further X’s at this point, and by (iv) 
they must be followed by an S, so again the permutations are different. 


11. Proceeding as in exercise 4, we let gnm be the number of partial admissible 
sequences of length n, leaving m elements on the deque, not ending in the symbol X; 
hnm is defined analogously, for those sequences that do end with X. We have 9(n+1)m = 
29n(m—1) + An(m—1[M>1], and hinziym = In(m+1) + Mn(m4i)- Define G(x, z) and 
H (a, z) by analogy with the definition in exercise 4; we have 


G(x, z) = wz + 2072? + 4a? 2? + (82 + 2x?) 2* + (162° + 803) 2° +---; 
5 


H (a, z) = 27 + 2x29 + (40? + 2)2* + (807 +62) +. 
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Setting h(z) = H(0, z), we find z~'G(a,z) = 2aG(a, z) + x(H (x, z) — h(z)) + x, and 
z'H(a,z) =a 'G(a,z) +a71(A(a, z) — h(z)); consequently 
xz(x — z — zh(z)) 


G = ; 
(z, 2) z — z — 2g2z + x22 


As in exercise 4, we try to choose h(z) so that the numerator cancels with a factor of 
the denominator. We find G(x, z) = xz/(1 — 2xr2(z)) where 


(2+1 (22 + 1)? — 822). 


Using the convention bo = 1, the desired generating function comes to 


3(3-—2- V1— 624+ 22) =14+24227 +629 +2224 +9025 +- 


By differentiation we find a recurrence relation that is handy for calculation: nbn = 
3(2n — 3)bn-1 — (n — 3)bn—2, n > 2. 

Another way to solve this problem, suggested by V. Pratt, is to use context-free 
grammars for the set of strings (see Chapter 10). The infinite grammar with produc- 
tions S > q"(Bx)", B + sq"(Bx)"*'B, for all n > 0, and B —> e, is unambiguous, 
and it allows us to count the number of strings with n x’s, as in exercise 2.3.4.4-31. 
12. We have an = 4” /V nn? + O(4"n~*/?) by Stirling’s formula. To analyze bn, let 
us first consider the general problem of estimating the coefficient of w” in the power 
series for VI — w yI — aw when |a| < 1. We have, for sufficiently small a, 


V1l—wV1—aw = V1—-wV1-ata(1—w) = V1 a D(a- u, 


where 8 = a/(1 — a); hence the desired coefficient is (—1)” V1 — a >>, (1/2) BP (po), 
Now 


cay (h it?) = oo) _  P(n—k—-1/2) o (=1/2)= T 
n E n T'(n+1)IP(—k-1/2) Jain , 
and n7471 = Ppa wre: k-1/2-j 4 Q(n—*-8/2—™) by Eq. 1.2.11.1-(16). Thus 


we obtain the asymptotic series [w"] VI = w/1— aw = con@*/? + cin? +--+ 
Cmn—™—3/2 4 O(n-™—5/2) where 


_ [Ma api afi Pie a 
G= NT > k a a ae 
For bn, we write 1 — 6z + 2° = (1 — (3 + V8)z)(1— (3 — V8)z) and let w = (3 + V8)z, 


a = (3 — V8)/(3+ V8), obtaining the asymptotic formula 


(v2 — DB + v8)" 4 + O(n-?)) = (V2 +1)??? 
93/471/23/2 + Of ) 93/4q71/23/2 


bn = (1+ O(n7*)). 


13. V. Pratt has found that a permutation is unobtainable if and only if it contains a 
subsequence whose relative magnitudes are respectively 
5,2,7,4,...,4k4+1,4k—-2,3,4k,1 or 5,2,7,4,...,44+3,4k,1,4k+2,3 


for some k > 1, or the same with the last two elements interchanged, or with the 1 
and 2 interchanged, or both. Thus the forbidden patterns for k = 1 are 52341, 52314, 
51342, 51324, 5274163, 5274136, 5174263, 5174236. [STOC 5 (1973), 268-277.] 
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14. (Solution by R. Melville, 1980.) Let R and S' be stacks such that the queue runs 
from top to bottom of R followed by bottom to top of S. When R is empty, pop the 
elements of S onto R until S becomes empty. To delete from the front, pop the top 
of R, which will not be empty unless the entire queue is empty. To insert at the rear, 
push onto S' (unless R is empty). Each element is pushed at most twice and popped 
at most twice before leaving the queue. 


SECTION 2.2.2 


1. M— 1 (not M). If we allowed M items, as (6) and (7) do, it would be impossible to 
distinguish an empty queue from a full one by examination of R and F, since only M 
possibilities can be detected. It is better to give up one storage cell than to make the 
program overly complicated. 


2. Delete from rear: If R = F then UNDERFLOW; Y + X[R]; if R = 1 then R + M, 
otherwise R + R — 1. Insert at front: Set X[F] + Y; if F = 1 then F + M, otherwise 
F + F — 1; if F = R then OVERFLOW. 


3. (a) LD1 I; LDA BASE,7:1. This takes 5 cycles instead of 4 or 8 as in (8). 
(b) Solution 1: LDA BASE,2:7 where each base address is stored with Iı = 0, 
I2 = 1. Solution 2: If it is desired to store the base addresses with Iı = I2 = 0, we 
could write LDA X2,7:1 where location X2 contains NOP BASE,2:7. The second solution 
takes one more cycle, but it allows the base table to be used with any index registers. 
(c) This is equivalent to ‘LD4 X(0:2)’, and takes the same execution time, except 
that rI4 will be set to +0 when X(0:2) contains —0. 
4. (a) NOP *,7. (b) LDA X,7:7(0:2). (c) This is impossible; the code LDA Y,7:7 
where location Y contains NOP X,7:7 breaks the restriction on 7:7. (See exercise 5.) 
(d) LDA X,7:1 with the auxiliary constants 


X NOP ¥*+1,7:2 
NOP *+1,7:3 
NOP ¥*+1,7:4 
NOP 0,5:6 


The execution time is 6 units. (e) INC6 X,7:6 where X contains NOP 0,6:6. 
5. (a) Consider the instruction ENTA 1000,7:7 with the memory configuration 
location ADDRESS Ij 


Fi 
N 


1000: 1001 7 7 
1001: 1004 7T 1 
1002: 1002 2 2 
1003: 1001 1 1 
1004: 1005 1 7 
1005: 1006 1 7 
1006: 1008 7 7 
1007: 1002 7 1 
1008: 1003 7 2 


and with rll = 1, rl2 = 2. We find that 1000,7,7 = 1001,7,7,7 = 1004,7,1,7,7 = 
1005,1,7,1,7,7 = 1006,7,1,7,7 = 1008,7,7,1,7,7 = 1003,7,2,7,1,7,7 = 1001,1,1,2,7,1,7,7 
= 1002,1,2,7,1,7,7 = 1003,2,7,1,7,7 = 1005,7,1,7,7 = 1006,1,7,1,7,7 = 1007,7,1,7,7 = 
1002,7,1,1,7,7 = 1002,2,2,1,1,7,7 = 1004,2,1,1,7,7 = 1006,1,1,7,7 = 1007,1,7,7 = 1008,7,7 
= 1003,7,2,7 = 1001,1,1,2,7 = 1002,1,2,7 = 1003,2,7 = 1005,7 = 1006,1,7 = 1007,7 = 
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1002,7,1 = 1002,2,2,1 = 1004,2,1 = 1006,1 = 1007. (A faster way to do this derivation 
by hand would be to evaluate successively the addresses specified in locations 1002, 
1003, 1007, 1008, 1005, 1006, 1004, 1001, 1000, in this order; but a computer evidently 
needs to go about the evaluation essentially as shown.) The author tried out several 
fancy schemes for changing the contents of memory while evaluating the address, with 
everything to be restored again by the time the final address has been obtained. Similar 
algorithms appear in Section 2.3.5. However, these attempts were unfruitful and it 
appears that there is just not enough room to store the necessary information. 

(b,c) Let H and C be auxiliary registers and let N be a counter. To get the effective 
address M, for the instruction in location L, do the following: 


A1. [Initialize.] Set H + 0, C + L, N + 0. (In this algorithm, C is the “current” 
location, H is used to add together the contents of various index registers, and 
N measures the depth of indirect addressing.) 

A2. [Examine address.] Set M < ADDRESS(C). If Ia (C) = j, 1 < j < 6, set 
M 4 M + rIj. If I2 (C) = j, 1 < j < 6, set H <4 H + rIj. If I4 (C) = I2 (C) 

set N+ N+1,H& 0. 

A3. [Indirect?] If either Iı (C) or I2(C) equals 7, set C + M and go to A2. 
Otherwise set M 4 M + H, H & 0. 


A4. [Reduce depth.] If N > 0, set C + M, N + N — 1, and go to A2. Otherwise M is 
the desired answer. J 


This algorithm will handle any situation correctly except those in which I; = 7 
and 1 < Ig < 6 and the evaluation of the address in ADDRESS involves a case with 
Iı = I2 = 7. The effect is as if Iz were zero. To understand the operation of Algo- 
rithm A, consider the notation of part (a); the state “L,7,1,2,5,2,7,7,7,7” is represented 
by C or M = L, N = 4 (the number of trailing 7s), and H = rll + rl2 + rI5 + rI2 (the 
post-indexing). In a solution to part (b) of this exercise, the counter N will always be 
either 0 or 1. 


6. (c) causes OVERFLOW. (e) causes UNDERFLOW, and if the program resumes it causes 
OVERFLOW on the final Io. 
7. No, since TOP [i] must be greater than OLDTOP [i]. 


8. With a stack, the useful information appears at one end with the vacant informa- 


tion at the other: 
B C 


where A = BASE[j], B = TOPLj], C = BASE[7 + 1]. With a queue or deque, the useful 
information appears at the ends with the vacant information somewhere in the middle: 


A B C D 


or in the middle with the vacant information at the ends: 
A C B D 


where A = BASE[j], B = REAR[j], C = FRONT[j], D = BASE[j +1]. The two cases 
are distinguished by the conditions B < C and B > C, respectively, in a nonempty 
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queue; or, if the queue is known not to have overflowed, the distinguishing conditions 
are respectively B < C and B > C. The algorithms should therefore be modified in an 
obvious way so as to widen or narrow the gaps of vacant information. (Thus in case 
of overflow, when B = C, we make empty space between B and C by moving one part 
and not the other.) In the calculation of SUM and D[j] in step G2, each queue should 
be considered to occupy one more cell than it really does (see exercise 1). 


9. Given any sequence specification a1,a2,...,@m there is one move operation re- 
quired for every pair (j,k) such that j < k and aj > ax. (Such a pair is called an 
“inversion”; see Section 5.1.1.) The number of such pairs is therefore the number of 
moves required. Now imagine all n™ specifications written out, and for each of the 
(2) pairs (j,k) with j < k count how many specifications have aj > ag. Clearly this 
equals (2), the number of choices for aj and ax, times n™~? the number of ways to fill 
in the remaining places. Hence the total number of moves among all specifications is 
(5) (3)n™~?. Divide this by n” to get the average, Eq. (14). 


10. As in exercise 9 we find that the expected value is 


(2) rma 5 


1<j<k<n 


| 

jaa 
a ™~ 
v3 
h a 
— 
loam 
K 

= 

3 

3 
Ss 

N 
a 
3 

= 
3s 

3N 
WN 
— 


= 5 (7) )G- (i++ +78). 


For this model, it makes absolutely no difference what the relative order of the lists 
is! (A moment’s reflection explains why; if we consider all possible permutations of a 
given sequence a1,...,@m, we find that the total number of moves summed over all 
these permutations depends only on the number of pairs of distinct elements a; 4 ax.) 


11. Counting as before, we find that the expected number is 


Emnt = -i () 3 Sk- n(*~*\a— 1)F-2-7 


here r is the number of entries in a1, a2, ..., @x—1 that equal a,. This quantity can 
also be expressed in the simpler form 


OO OGO 


Is there a simpler way yet to give the answer? Apparently not, since the generating 
function for given n and t is 


m n-li z z t+ 
P Emme =- Croa (z+ (1 -= z)n(t+1)). 


12. If m = 2k, the average is 27°F times 


(7 )2k+ (Peet (e+ (ent t (37 )2e. 


The latter sum is 


2k 2k—1 2k—1 (2k fe fy 
(w )e+2(( k Jet (AT) = (Feta oe 
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A similar argument may be used when m = 2k+ 1. The answer is 


mm (m1 

2 2m \|m/2))° 
13. A. C. Yao has proved that we have Emax(ki, k2) = $m + (27(1— 2p)) P ym + 
O(m-'/?(logm)*) for large m, when p < +. [SICOMP 10 (1981), 398-403.] And 


P. Flajolet has extended the analysis, showing in particular that the expected value is 
asymptotically am when p = i, where 


-8 D sin(nz/2) cosh(n7/2) 


n?n? sinh nt 


& 0.67531 44833. 


a= 
n>1 


Moreover, when p > l the final value of kı tends to be uniformly distributed as m —> oo, 
so E max(kı, k2) ~ 2m. [See Lecture Notes in Comp. Sci. 233 (1986), 325-340.] 


14. Let kj = m/n + ym zj. (This idea was suggested by N. G. de Bruijn.) Stirling’s 
approximation implies that 
=i m! 


n Fal. gl maha, Kn) 


= (Vrm) n? ( z + ym max(zı,..., an) 


x exp (-F(2i ++++-+2%)) (/m)'” (1 +0(=)) 


when ki +---+kn = m and when the z’s are uniformly bounded. The sum of the latter 
quantity over all nonnegative ki,...,k, satisfying this condition is an approximation 
to a Riemann integral; we may deduce that the asymptotic behavior of the sum is 
an(m/n) + eni/m + O(1), where 


an = (vV) "n" f exp (-F(ei+- : -+22)) dx2...dxn, 
x44 


| .--tarm =0 


max(zı,..., £n) exp (-2ei+. : -+22)) dz... dEn, 


a= ( am) n" 
21-44 


L.--tatn =0 


since it is possible to show that the corresponding sums come within € of an and cn for 
any €. 

We know that an = 1, since the corresponding sum can be evaluated explicitly. 
The integral that appears in the expression for c, equals nI, where 


n 
n= foc sage 0 (Yeh tt) den dn 


Titan 


We may make the substitution 
1 
zı = z (u2 +: + Yn), £2 = Tı — Y2, T3 = T1 — Y3, «---, Ln = T1 — Yn; 
then we find I; = hjn’, where 


In =f (yo +- -+0n)exp(-$ ) dy2...dyn, 
y Yn 20 
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and Q = n(y3 +--+: + y2) — (y2 +--+ yn). Now by symmetry, Iz is (n — 1) times the 
same integral with (y2+---+ yn) replaced by y2; hence I2 = (n — 1)I3, where 
= | (nya = (2 +++ + yn)) exp(-) dyz... dyn 
Y2- Yn Z0 


ai exp(-2) dys . . . dYn; 
Y3 Yn Z0 2 


kedagi 


here Qo is Q with y2 replaced by zero. [When n = 2, let I3 = 1.] Now let z; = 
Vnyj— (ys + +yn)/(V2+v7n),3 <j <n. Then Qo = 23+: -+ z2, and we deduce 
that Is = Ia [nY V2, where 


2 2 
n= f exp (==) dz3...dzn 
Y3 Ynz0 2 


ieai 


2 2 
= an f exp (=) dz3 ...d2n = An(V Qn)”, 


where a, is the “solid angle” in (n — 2)-dimensional space spanned by the vectors 
(n+V2n, 0,...,0) —(1,1,...,1), ..., (0,0,...,n+V2n) — (1,1,...,1), divided by 
the total solid angle of the whole space. Hence 
(n-1)Vn 

2/nr 


We have az = 1, ag L, as = n`! arctan V2 = .304, and 


Cn = 


as = : + Ž arctan x = .206. 
[The value of c3 was found by Robert M. Kozelka, Annals of Math. Stat. 27 (1956), 
507-512, but the solution to this problem for higher values of n has apparently never 
appeared in the literature.] 


16. Not unless the queues meet the restrictions that apply to the primitive method of 


(4) and (5). 

17. First show that BASE[7], < BASE[7], at all times. Then observe that each overflow 
for stack i in so(a) that does not also overflow in sı(ø) occurs at a time when stack i 
has gotten larger than ever before, yet its new size is not more than the original size 
allocated to stack i in s1(¢). 


18. Suppose the cost of an insertion is a, plus bN + cn if repacking is needed, where 
N is the number of occupied cells; let the deletion cost be d. After a repacking that 
leaves N cells occupied and S = M — N cells vacant, imagine that each insertion until 
the next repacking costs a + b + 10c + 10(b + ¢)nN/S = O(1 + na/(1 — a)), where 
a = N/M. If p insertions and q deletions occur before that repacking, the imagined 
cost is p(a+b+ 10c+10(b+c)nN/S) + qd, while the actual cost is pa+bN’+cn+qd < 
pa+pb+bN + cn + qd. The latter is less than the imagined cost, because p > .15/n; 
our assumption that M > n? implies that cS/n + (b+ c)N > bN + cn. 


19. We could simply decrease all the subscripts by 1; the following solution is slightly 
nicer. Initially T = F =R = 0. 

Push Y onto stack X: If T = M then OVERFLOW; X[T] + Y; T+ T+1. 

Pop Y from stack X: If T = 0 then UNDERFLOW; T 4+ T — 1; Y + X[T]. 


2.2.3 ANSWERS TO EXERCISES 545 


Insert Y into queue X: X[R] + Y; R + (R + 1) mod M; if R = F then OVERFLOW. 

Delete Y from queue X: if F = R then UNDERFLOW; Y + X[F]; F + (F + 1) mod M. 
As before, T is the number of elements on the stack, and (R — F) mod M is the number 
of elements on the queue. But the top stack element is now X[T — 1], not X[T]. 

Even though it is almost always better for computer scientists to start counting 
at 0, the rest of the world will probably never change to 0-origin indexing. Even Edsger 
Dijkstra counts “1-2-3-4 | 1-2-3-4” when he plays the piano! 


SECTION 2.2.3 
1. OVERFLOW is implicit in the operation P <= AVAIL. 
2. INSERT STJ 1F Store location of ‘NOP T. 
STJ 9F Store exit location. 
LD1 AVAIL rll < AVAIL. 


J1Z OVERFLOW 
LD3 0,1(LINK) 


ST3 AVAIL 
STA 0,1(INFO) INFO(I1) + Y. 
1H LD3 (0:2) rl3 + LOC(T). 
LD2 0,3 rl2 + T. 
ST2 0,1(LINK) LINK(rIl) <T. 
ST1 0,3 T& rll. 
9H JMP * l 
3. DELETE STJ 1F Store location of ‘NOP T. 
STJ 9F Store exit location. 
1H LD2 (0:2) rl2 + LOC (T). 
LD3 0,2 rl3 +T. 
J3Z 9F Is T = 4? 
LD1 0,3(LINK) rll + LINK(T). 
ST1 0,2 T& rll. 
LDA 0,3(INFO) rA + INFO(rI1). 
LD2 AVAIL AVAIL < rI3. 
ST2 0,3(LINK) 
ST3 AVAIL 
ENT3 2 Prepare for second exit. 
9H JMP *,3 I 
4. OVERFLOW STJ 9F Store setting of rJ. 
ST1 8F(0:2) Save rI1 setting. 
LD1 POOLMAX 
ST1 AVAIL Set AVAIL to new location. 
INCi c 
ST1 POOLMAX Increment POOLMAX. 
CMP1 SEQMIN 
JG TOOBAD Has storage been exceeded? 
STZ -c,1(LINK) Set LINK(AVAIL) + A. 
9H ENT1 * Take rJ setting. 
DEC1 2 Subtract 2. 
STi ¥*+2(0:2) Store exit location. 
8H ENT1 * Restore rll. 


JMP * Return. J 
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5. Inserting at the front is essentially like the basic insertion operation (8), with an 
additional test for empty queue: P <= AVAIL, INFO(P) + Y, LINK(P) + F; if F = A then 
R&P; FeP. 

To delete from the rear, we would have to find which node links to NODE(R), and 
that is necessarily inefficient since we have to search all the way from F. This could be 
done, for example, as follows: 

a) If F = A then UNDERFLOW, otherwise set P <— LOC(F). 

b) If LINK(P) ÆR then set P + LINK(P) and repeat this step until LINK(P) =R. 

c) Set Y < INFO(R), AVAIL < R, R + P, LINK(P) + A. 


6. We could remove the operation LINK(P) + A from (14), if we delete the commands 
“F < LINK(P)” and “if F = A then set R «+ LOC(F)” from (17); the latter are to be 
replaced by “if F = R then F + A and R + LOC(F), otherwise set F + LINK(P)”. 

The effect of these changes is that the LINK field of the rear node in the queue will 
contain spurious information that is never interrogated by the program. A trick like 
this saves execution time and it is quite useful in practice, although it violates one of 
the basic assumptions of garbage collection (see Section 2.3.5) so it cannot be used in 
conjunction with such algorithms. 


7. (Make sure that your solution works for empty lists.) 
I1. Set P + FIRST, Q + A. 
I2. If P Æ A, set R + Q, Q + P, P+ LINK(Q), LINK(Q) + R, and repeat this step. 
I3. Set FIRST Q. I 


In essence we are popping nodes off one stack and pushing them onto another. 


8. LD1 FIRST 1 Il. P=rlIl + FIRST. 
ENT2 0 1 Q=rQ¢A. 
JiZ 2F 1 I2. If the list is empty, jump. 
1H ENTA 0,2 n R=rA¢Q. 
ENT2 0,1 n Q¢eP. 
LD1 0,2(LINK) n P+¢ LINK(Q). 
STA 0,2(LINK) n LINK(Q) < R. 
J1iNZ 1B n IsP# <A? 
2H ST2 FIRST 1 I3. FIRST+ Q. J 


The time is (72 + 6)u. Better speed (5n + constant)u is attainable; see exercise 1.1-3. 


9. (a) Yes. (b) Yes, if biological parenthood is considered; no, if legal parenthood 
is considered (a man’s daughter might marry his father, as in the song “I’m My Own 
Grampa”). (c) No (—1 < 1and1 < —1). (d) Let us hope so, or else there is a circular 
argument. (e) 1 <3 and3~< 1. (f) The statement is ambiguous. If we take the 
position that the subroutines called by y are dependent upon which subroutine calls y, 
we would have to conclude that the transitive law does not necessarily hold. (For 
example, a general input-output subroutine might call on different processing routines 
for each I/O device present, but these processing subroutines are usually not all needed 
in a single program. This is a problem that plagues many automatic programming 
systems.) 


10. For (i) there are three cases: z = y; x C y and y = z; x C y and y C z. For (ii) 
there are two cases: x = y; x # y. Each case is handled trivially, as is (iii). 

11. “Multiply out” the following to get all 52 solutions: 13749(25 + 52)86 + (1379 
1397 + 1937 + 9137) (4258 + 4528 + 2458 + 5428 + 2548 + 5248 + 2584 + 5284)6 + (1392 
1932 + 1923 + 9123 + 9132 + 9213)7(458 + 548 + 584)6. 
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12. For example: (a) List all sets with k elements (in any order) before all sets with 
k +1 elements, 0 < k < n. (b) Represent a subset by a sequence of 0s and 1s showing 
which elements are in the set. This gives a correspondence between all subsets and the 
integers 0 through 2” — 1, via the binary number system. The order of correspondence 
is a topological sequence. 


13. Sha and Kleitman, Discrete Math. 63 (1987), 271-278, have proved that the 


number is at most J [p-o a, This exceeds the obvious lower bound |];_y (%)! = 
peter OueEs)) by a factor of e? FOl), they conjecture that the lower bound is closer 


to the truth. 


14. If a1a2...an and bı b2...bn are two possible topological sorts, let j be minimal 
such that aj # bj; then ax = bj and aj = bm for some k,m > j. Now bj £ aj since 
k > j, and aj £ bj since m > j, hence (iv) fails. Conversely if there is only one 
topological sort aı a2 ...an, we must have aj < aj+1 for 1 < j < n, since otherwise aj 
and aj+ı could be interchanged. This and transitivity imply (iv). 

Note: The following alternative proofs work also for infinite sets. (a) Every partial 
ordering can be embedded in a linear ordering. For if we have two elements with xo A yo 
and yo £ Xo we can generate another partial ordering by the rule “x < y or (x < zo and 
yo x y).” The latter ordering “includes” the former and has zo < yo. Now apply Zorn’s 
lemma or transfinite induction in the usual way to complete the proof. (b) Obviously 
a linear ordering cannot be embedded in any different linear ordering. (c) A partial 
ordering that has incomparable elements xp and yo as in (a) can be extended to two 
linear orderings in which xo < yo and yo < zo, respectively, so at least two linear 
orderings exist. 


15. If S is finite, we can list all relations a < b that are true in the given partial 
ordering. By successively removing, one at a time, any relations that are implied by 
others, we arrive at an irredundant set. The problem is to show there is just one such 
set, no matter in what order we go about removing redundant relations. If there were 
two irredundant sets U and V, in which “a < b” appears in U but not in V, there are 
k +1 relations a < cı < --- < ck < bin V for some k > 1. But it is possible to deduce 
a < cı and cı < b from U, without using the relation a < b (since b £ cı and ci Z a), 
hence the relation a < b is redundant in U. 

The result is false for infinite sets S, when there is at most one irredundant set 
of relations. For example if S denotes the integers plus the element oo and we define 
n~<n+1landn < œ for all n, there is no irredundant set of relations that characterizes 
this partial ordering. 

16. Let rp, Lp. ...Xp,, be a topological sorting of S; apply the permutation pip2...Dn 
to both rows and columns. 

17. If k increases from 1 to n in step T4, the output is 1932745860. If k decreases 
from n to 1 in step T4, as it does in Program T, the output is 9123745860. 

18. They link together the items in sorted order: QLINK[O] first, QLINK[QLINK[0]] 
second, and so on; QLINK[last] = 0. 

19. This would fail in certain cases; when the queue contains only one element in 
step T5, the modified method would set F = 0 (thereby emptying the queue), but other 
entries could be placed in the queue in step T6. The suggested modification would 
therefore require an additional test of F = 0 in step T6. 


20. Indeed, a stack could be used, in the following way. (Step T7 disappears.) 
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T4. 


T5. 


T6. 


Set T + 0. For 1 < k < n if COUNT[k] is zero do the following: Set 
SLINK[k] + T, T4 k. (SLINK[k] = QLINK[K].) 

Output the value of T. If T=0, go to T8; otherwise, set N 4+ N — 1, P 4+ TOP [T], 
T< SLINK[T]. 

Same as before, except go to T5 instead of T7; and when COUNT [SUC(P)] goes 
down to zero, set SLINK[SUC(P)] « T and T + SUC (P). 


21. Repeated relations only make the algorithm a little slower and take up more space 
in the storage pool. A relation “j < j” would be treated like a loop (an arrow from a 
box to itself in the corresponding diagram), which violates partial order. 


22. To make the program “fail-safe” we should (a) check that 0 < n < some appropri- 
ate maximum; (b) check each relation j < k for the conditions 0 < j,k < n; (c) make 
sure that the number of relations doesn’t overflow the storage pool area. 

23. At the end of step T5, add “TOP[F] + A”. (Then at all times TOP[1],...,TOP [n] 
point to all the relations not yet canceled.) In step T8, if N > 0, print ‘LOOP DETECTED 
IN INPUT:’, and set QLINK[k] + 0 for 1 < k < n. Now add the following steps: 


T9 


T10. 


T11. 
T12. 
T13. 


T14. 


. For 1 < k < n set P + TOP[K], TOP[k] + 0, and perform step T10. (This 
will set QLINK[7] to one of the predecessors of object j, for each j not yet 
output.) Then go to T11. 


If P Æ A, set QLINK[SUC(P)] + k, P + NEXT(P), and repeat this step. 


Find a k with QLINK[k] 4 0. 

Set TOP[A] «+ 1 and k + QLINK[k]. Now if TOP[k] = 0, repeat this step. 
(We have found the start of a loop.) Print the value of k, set TOP[k] + 0, 
k + QLINK[k], and if TOP[k] = 1 repeat this step. 

Print the value of k (the beginning and end of the loop) and stop. (Note: 
The loop has been printed backwards; if it is desired to print the loop in 


forward order, an algorithm like that in exercise 7 should be used between 
steps T12 and T13.) I 


24. Insert three lines in the program of the text: 
08a PRINTER EQU 18 


14a 
59a 


Repl 


T9 


T10 


ST6 NO 

STZ X,1(TOP) TOP[F] + A. 
ace lines 74-75 by the following: 

J6Z DONE 

OUT LINE1(PRINTER) Print indication of loop. 

LD6 NO 

STZ X,6(QLINK) QLINK[k] + 0. 

DEC6 1 

J6P ¥*-2 n>k>1. 

LD6 NO 

LD2 X,6(TOP) P + TOP[K]. 

STZ X,6(TOP) TOP[k] < 0. 

J2Z T9A Is P= A? 

LD1 0,2(SUC) rll + SUC(P). 

ST6 X,1(QLINK) QLINK[rI1] < k. 


LD2 0,2(NEXT) P + NEXT(P). 
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87 J2P T10 Is P # A? 
88 T9A DEC6 1 
89 J6P T9 n>k>1. 
90 Tii INC6 1 
91 LDA X,6(QLINK) 
92 JAZ *-2 Find k with QLINK[k] Æ 0. 
93 T12 ST6 X,6(TOP) TOP[k] < k. 
94 LD6 X,6(QLINK) k + QLINK[K]. 
95 LD1 X,6(TOP) 
96 JiZ T12 Is TOP [k] = 0? 
97 T13 ENTA 0,6 
98 CHAR Convert k to alphameric. 
99 JBUS *(PRINTER) 
100 STX VALUE Print. 
101 OUT LINE2(PRINTER) 
102 JiZ DONE Stop when TOP[k] = 0. 
108 STZ X,6(TOP) TOP[k] <0. 
104 LD6 X,6(QLINK) k + QLINK[K]. 
105 LD1 X,6(TOP) 
106 JMP T13 
107 LINE1 ALF LOOP Title line 
108 ALF DETEC 
109 ALF TED I 
110 ALF N INP 
111 ALF UT: 
112 LINE2 ALF Succeeding lines 
113 VALUE EQU LINE2+3 
114 ORIG LINE2+24 
115 DONE HLT End of computation 
116 X END TOPSORT I 


Note: If the relations 9 < 1 and 6 < 9 are added to the data (18), this program 
will print “9, 6, 8, 5, 9” as the loop. 


26. One solution is to proceed in two phases as follows: 
Phase 1. (We use the X table as a (sequential) stack as we mark B = 1 or 2 for each 
subroutine that needs to be used.) 

AO. For 1 < J < N set BCX[J]) + B(X[J]) + 2, if B(X[J]) < 0. 

A1. If N = 0, go to phase 2; otherwise set P + X[N] and decrease N by 1. 

A2. If |B(P)| = 1, go to A1, otherwise set P + P + 1. 


A3. If B(SUB1(P)) < 0, set N + N + 1, B(SUB1(P)) «+ B(SUB1(P)) + 2, X[N] + 
SUB1(P). If SUB2(P) # 0 and B(SUB2(P)) < 0, do a similar set of actions with 
SUB2 (P). Go to A2. J 


Phase 2. (We go through the table and allocate memory.) 
B1. Set P + FIRST. 


B2. If P = A, set N+ N+ 1, BASE(LOC(X[N])) < MLOC, SUB(LOC(X[N])) < 0, and 
terminate the algorithm. 


550 ANSWERS TO EXERCISES 2.2.3 


B3. If B(P) > 0, set N 4+ N + 1, BASE(LOC(X[N])) 4+ MLOC, SUB(LOC(X[N])) +P, 
MLOC + MLOC + SPACE (P). 


B4. Set P + LINK(P) and return to B2. I 


27. Comments on the following code are left to the reader. 


B EQU 0:1 A1 JiZ Bil INC1 1 
SPACE EQU 2:3 LD2 X,1 INCA 2 
LINK EQU 4:5 DEC1 1 STA 0,3(B) 
SUB1 EQU 2:3 A2 LDA 0,2(1:1) ST3 X,1 
SUB2 EQU 4:5 DECA 1 JMP A2 
BASE EQU 0:3 JAZ Al B1 ENT2 FIRST 
SUB EQU 4:5 INC2 1 LDA MLOC 
AO LD2 N A3 LD3 0,2(SUB1) JMP 1F 
J2Z 2F LDA 0,3(B) B3 LDX 0,2(B) 
1H LD3 X,2 JAP 9F JXNP B4 
LDA 0,3(B) INC1 1 INC1 1 
JAP *+3 INCA 2 ST2 X,1(SUB) 
INCA 2 STA 0,3(B) ADD 0,2(SPACE) 
STA 0,3(B) ST3 X,1 1H STA X+1,1(BASE) 
DEC2 1 9H LD3 0,2(SUB2) B4 LD2 0,2(LINK) 
J2P 1B J3Z A2 B2 J2NZ B3 
2H LD1 N LDA 0,3(B) STZ X+1,1(SUB) I 
JAP A2 


28. We give here only a few comments related to the military game. Let A be the 
player with three men whose pieces start on nodes A13; let B be the other player. In 
this game, A must “trap” B, and if B can cause a position to be repeated for a second 
time we can consider B the winner. To avoid keeping the entire past history of the 
game as an integral part of the positions, however, we should modify the algorithm in 
the following way: Start by marking the positions 157—4, 789-B, 359-6 with B to move 
as “lost” and apply the suggested algorithm. Now the idea is for player A to move 
only to B’s lost positions. But A must also take precautions against repeating prior 
moves. A good computer game-playing program will use a random number generator 
to select between several winning moves when more than one is present, so an obvious 
technique would be to make the computer, playing A, just choose randomly among 
those moves that lead to a lost position for B. But there are interesting situations 
that make this plausible procedure fail! For example, 
consider position 258-7 with A to move; this is a 
won position. From position 258-7, player A might 
try moving to 158-7 (which is a lost position for B, 
according to the algorithm). But then B plays to 
158-B, and this forces A to play to 258-B, after which 
B plays back to 258-7; B has won, since the former 
position has been repeated! This example shows that 
the algorithm must be re-invoked after every move 
has been made, starting with each position that has 
previously occurred marked “lost” (if A is to move) or 
“won” (if B is to move). The military game makes a 
very satisfactory computer demonstration program. 


Board for the “military game.” 


2.2.4 ANSWERS TO EXERCISES 551 


29. (a) If FIRST = A, do nothing; otherwise set P + FIRST, and then repeatedly set 
P + LINK(P) zero or more times until LINK(P) = A. Finally set LINK(P) + AVAIL and 
AVAIL + FIRST (and probably also FIRST + A). (b) If F = A, do nothing; otherwise 
set LINK(R) + AVAIL and AVAIL + F (and probably also F + A, R + LOC(F)). 


30. To insert, set P < AVAIL, INFO(P) + Y, LINK(P) + A, if F = A then F + P 
else LINK(R) + P, and R + P. To delete, do (9) with F replacing T. (Although it 
is convenient to let R be undefined for an empty queue, this lack of discipline might 
confuse a garbage collection algorithm, as in exercise 6.) 


SECTION 2.2.4 


1. No, it does not help; it seems to hinder, if anything. (The stated convention is not 
especially consistent with the circular list philosophy, unless we put NODE (LOC (PTR) ) 
into the list as its list head.) 


2. Before: PTR, PTR2 


After: l i. 


(IPLIPliPliplipl yl t 


3. If PTR: = PTR2, the only effect is PTR2 «+ A. If PTR: Æ PTR2, the exchange of links 
breaks the list into two parts, as if a circle had been broken in two by cutting at two 
points; the second part of the operation then makes PTR, point to a circular list that 
consists of the nodes that would have been traversed if, in the original list, we followed 
the links from PTR; to PTR. 


4. Let HEAD be the address of the list head. To push down Y onto the stack: Set 
P < AVAIL, INFO(P) 4+ Y, LINK(P) 4+ LINK(HEAD), LINK(HEAD) < P. To pop up the 
stack onto Y: If LINKC(HEAD) = HEAD then UNDERFLOW; otherwise set P « LINK(HEAD), 
LINK(HEAD) + LINK(P), Y + INFO(P), AVAIL <P. 


5. (Compare with exercise 2.2.3-7.) Set Q + A, P + PTR, and then while P # A 
repeatedly set R + Q, Q + P, P + LINK(Q), LINK(Q) + R. (Afterwards Q = PTR.) 


a; 


(b) -lolo|1 


7. Matching terms in the polynomial are located in one pass over the list, so repeated 
random searches are avoided. Also, increasing order would be incompatible with the 
“—1” sentinel. 


8. We must know what node points to the current node of interest, if we are going to 
delete that node or to insert another one ahead of it. There are alternatives, however: 
We could delete NODE(Q) by setting Q2 +— LINK(Q) and then setting NODE(Q) + 
NODE(Q2), AVAIL < Q2; we could insert a NODE(Q2) in front of NODE(Q) by first 
interchanging NODE(Q2) + NODE(Q), then setting LINK(Q) + Q2, Q + Q2. These 
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clever tricks allow the deletion and insertion without knowing which node links to 
NODE(Q); they were used in early versions of IPL. But they have the disadvantage that 
the sentinel node at the end of a polynomial will occasionally move, and other link 
variables may be pointing to this node. 


9. Algorithm A with P = Q simply doubles polynomial(Q), as it should — except 
in the anomalous case that COEF = 0 for some term with ABC > 0, when it fails 
badly. Algorithm M with P = M also gives the expected result. Algorithm M with 
P = Q sets polynomial(P) < polynomial(P) times (1 + t1)(1 + t2)... (1 + tx) if 
M = tı + t2 +---+ tk (although this is not immediately obvious). When M = Q, 
Algorithm M surprisingly gives the expected result, polynomial(Q) + polynomial(Q) 
+ polynomial(Q) x polynomial(P), except that the computation blows up when the 
constant term of polynomial (P) is —1. 


10. None. (The only possible difference would be in step M2, removing error checks 
that A, B, or C might individually overflow; these error checks were not specified because 
we assumed that they were not necessary.) In other words, the algorithms in this section 
may be regarded as operations on the polynomial f (22, x?, x) instead of on f(x,y,z). 


11. COPY STJ 9F (comments ST6 1,3(LINK) 
ENT3 9F are ENT3 0,6 
LDA 1,1 left LD1 1,1(LINK) 

1H LD6 AVAIL to LDA 1,1 

J6Z OVERFLOW the JANN 1B 
LDX 1,6(LINK) reader) LD2 8F(LINK) 
STX AVAIL ST2 1,3(LINK) 
STA 1,6 9H JMP * 
LDA 0,1 8H CON O I 
STA 0,6 


12. Let the polynomial copied have p terms. Program A takes (27p + 13)u, and to 
make it a fair comparison we should add the time to create a zero polynomial, say 18u 
with exercise 14. The program of exercise 11 takes (21p + 31)u, about 78% as much. 


13. ERASE STJ 9F 
LDX AVAIL 
LDA 1,1(LINK) 
STA AVAIL 
STX 1,1(LINK) 
9H JMP * J 


14. ZERO STJ 9F MOVE 1F(2) 
LDi AVAIL ST2 1,2(LINK) 
J1Z OVERFLOW 9H JMP * 
LDX 1,1 (LINK) 1H CON 0O 
STX AVAIL CON -1(ABC) J 
ENT2 0,1 

15. MULT STJ 9F Entrance to subroutine 
LDA 5F Change settings of switches 
STA SW1 
LDA 6F 
STA SW2 


STA SW3 
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JMP *+2 
2H JMP ADD M2. Multiply cycle. 
1H LD4 1,4(LINK) M1. Next multiplier. M <4— LINK (M). 
LDA 1,4 
JANN 2B To M2 if ABC(M) > 0. 
8H LDA 7F Restore settings of switches. 
STA SW1 
LDA 8F 
STA SW2 
STA SW3 
9H JMP * Return. 
5H JMP *+1 New setting of SW1 
LDA 0,1 
MUL 0,4 rX + COEF (P) x COEF (M). 
LDA 1,1(ABC) ABC(P) 
JAN *+2 
ADD 1,4(ABC) + ABC (M), if ABC(P) > 0. 
SLA 2 Move into 0:3 field of rA. 
STX OF Save rX for use in SW2 and SW3. 
JMP SW1+1 
6H LDA OF New setting of SW2 and SW3 
7H LDA 1,1 Usual setting of SW1 
8H LDA 0,1 Usual setting of SW2 and SW3 
OH CON 0 Temp storage J 


16. Let r be the number of terms in polynomial(M). The subroutine requires 21pr + 
38r +29+27% m +185 m” +27 p' +8) units of time, where the summations 
refer to the corresponding quantities during the r activations of Program A. The 
number of terms in polynomial (Q) goes up by p'— m’ each activation of Program A. If 
we make the not unreasonable assumption that m’ = 0 and p' = ap where 0 < a < 1, 
we get the respective sums equal to 0, (1 — a)pr, apr, and rq) + ap(r(r — 1)/2), where 
qb is the value of q’ in the first iteration. The grand total is 4apr? + 40pr + 4apr + 
8qor + 38r + 29. This analysis indicates that the multiplier ought to have fewer terms 
than the multiplicand, since we have to skip over unmatching terms in polynomial (Q) 
more often. (See exercise 5.2.3-29 for a faster algorithm.) 


17. There actually is very little advantage; addition and multiplication routines with 
either type of list would be virtually the same. The efficiency of the ERASE subroutine 
(see exercise 13) is apparently the only important difference. 


18. Let the link field of node x; contain LOC (zi+1) 6 LOC(2;_1), where “@” denotes 
“exclusive or.” Other invertible operations, such as addition or subtraction modulo the 
pointer field size, could also be used. It is convenient to include two adjacent list heads 
in the circular list, to help get things started properly. (The origin of this ingenious 
technique is unknown.) 


SECTION 2.2.5 


1. Insert Y at the left: P < AVAIL; INFO(P) + Y; LLINK(P) + A; RLINK(P) + LEFT; 
if LEFT Æ A then LLINK(LEFT) + P else RIGHT + P; LEFT + P. Set Y to leftmost and 
delete: if LEFT = A then UNDERFLOW; P 4+ LEFT; LEFT 4+ RLINK(P); if LEFT = A then 
RIGHT < A, else LLINK(LEFT) + A; Y < INFO(P); AVAIL <P. 
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2. Consider the case of several deletions (at the same end) in succession. After each 
deletion we must know what to delete next, so the links in the list must point away 
from that end of the list. Deletion at both ends therefore implies that the links must 
go both ways. On the other hand, exercise 2.2.4—18 explains how to represent two links 
in a single link field; in that way general deque operations are possible. 


3. To show the independence of CALLUP from CALLDOWN, notice for example that in 
Table 1 the elevator did not stop at floors 2 or 3 at time 0393-0444 although there were 
people waiting; these people had pushed CALLDOWN, but if they had pushed CALLUP the 
elevator would have stopped. 

To show the independence of CALLCAR from the others, notice that in Table 1, when 
the doors start to open at time 1378, the elevator has already decided to be GOINGUP. 
Its state would have been NEUTRAL at that point if CALLCAR[1] = CALLCAR[2] = 
CALLCAR[3] = CALLCAR[4] = 0, according to step E2, but in fact CALLCAR[2] and 
CALLCAR[3] have been set to 1 by users 7 and 9 in the elevator. (If we envision the 
same situation with all floor numbers increased by 1, the fact that STATE = NEUTRAL 
or STATE = GOINGUP when the doors open would affect whether the elevator would 
perhaps continue to go downward or would unconditionally go upward.) 


4. If a dozen or more people were getting out at the same floor, STATE might be 
NEUTRAL all during this time, and when E9 calls the DECISION subroutine this may set 
a new state before anyone has gotten in on the current floor. It happens very rarely 
indeed (and it certainly was the most puzzling phenomenon observed by the author 
during his elevator experiments). 


5. The state from the time the doors start to open at time 1063 until user 7 gets in 
at time 1183 would have been NEUTRAL, since there would have been no calls to floor 0 
and nobody on board the elevator. Then user 7 would set CALLCAR[2] + 1 and the 
state would correspondingly change to GOINGUP. 


6. Add the condition “if OUT < IN then STATE 4 GOINGUP; if OUT > IN then STATE Æ 
GOINGDOWN” to the condition “FLOOR = IN” in steps U2 and U4. In step E4, accept 
users from QUEUE[FLOOR] only if they are headed in the elevator’s direction, unless 
STATE = NEUTRAL (when we accept all comers). 

[Stanford’s math department has just such an elevator, but its users don’t actually 
pay much attention to the indicator lights; people tend to get on as soon as they can, 
regardless of direction. Why didn’t the elevator designers realize this, and design the 
logic accordingly by clearing both CALLUP and CALLDOWN? The whole process would be 
faster, since the elevator wouldn’t have to stop as often.] 


7. In line 227 this user is assumed to be in the WAIT list. Jumping to U4A makes sure 
that this assumption is valid. It is assumed that GIVEUPTIME is positive, and indeed 
that it is probably 100 or more. 


8. Comments are left to the reader. 
277 E8 DEC4 1 


278 ENTA 61 

279 JMP HOLDC 

280 LDA CALL,4(3:5) 
281 JAP 1F 

282 ENT1 -2,4 

283 JiZ 2F 


284 LDA CALL,4(1:1) 
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285 JAZ E8 
286 2H LDA CALL-1,4 
287 ADD CALL-2,4 
288 ADD CALL-3,4 
289 ADD CALL-4,4 
290 JANZ E8 
291 1H ENTA 23 
292 JMP E2A I 
9. 01 DECISION STJ 9F Store exit location. 
02 J5NZ 9F D1. Decision necessary? 
03 LDX ELEVi+2(NEXTINST) 
04 DECX E1 D2. Should doors open? 
05 JXNZ 1F Jump if elevator not at E1. 
06 LDA CALL+2 
07 ENT3 E3 Prepare to schedule E3, 
08 JANZ 8F if there is a call on floor 2. 
09 1H ENT1 -4 D3. Any calls? 
10 LDA CALL+4,1 Search for a nonzero call variable. 
11 JANZ 2F 
12 1H INC1 1 rl=j-4 
13 JiNP *-3 
14 LDA 9F(0:2) All CALL[j], 7 # FLOOR, are zero. 
15 DECA E6B Is exit location = line 250? 
16 JANZ 9F 
17 ENT1 -2 Set j + 2. 
18 2H ENT5 4,1 D4. Set STATE. 
19 DEC5 0,4 STATE + j — FLOOR. 
20 J5NZ *+2 
21 JANZ 1B j = FLOOR not allowed in general. 
22 JXNZ 9F D5. Elevator dormant? 
28 J5Z OF Jump if not at El or if j = 2. 
24 ENT3 E6 Otherwise schedule E6. 
25 8H ENTA 20 Wait 20 units of time. 
26 ST6 8F(0:2) Save rl6. 
27 ENT6 ELEV1 
28 ST3  2,6(NEXTINST) Set NEXTINST to E3 or E6. 
29 JMP HOLD Schedule the activity. 
30 8H ENT6 * Restore rl6. 
31 9H JMP * Exit from subroutine. | 


11. Initially let LINK[k] =0,1<k < n, and HEAD = —1. During a simulation step that 
changes V[k], give an error indication if LINK[k] 4 0; otherwise set LINK[k] <+ HEAD, 
HEAD + k and set NEWV[k] to the new value of V[k]. After each simulation step, set 
k + HEAD, HEAD + —1, and do the following operation repeatedly zero or more times 
until k < 0: set Vik] + NEWV [k], t + LINK[A], LINK[k] + 0, k & t. 

Clearly this method is readily adapted to the case of scattered variables, if we 
include a NEWV and LINK field in each node associated with a variable field V. 


12. The WAIT list has deletions from the left to the right, but insertions are sorted 
in from the right to the left (since the search is likely to be shorter from that side). 
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Also we delete nodes from all three lists in several places when we do not know the 
predecessor or successor of the node being deleted. Only the ELEVATOR list could be 
converted to a one-way list, without much loss of efficiency. 

Note: It may be preferable to use a nonlinear list as the WAIT list in a discrete 
simulator, to reduce the time for sorting in. Section 5.2.3 discusses the general problem 
of maintaining priority queues, or “smallest in, first out” lists, such as this. Several 
ways are known in which only O(log n) operations are needed to insert or delete when 
there are n elements in the list, although there is of course no need for such a fancy 
method when n is known to be small. 


SECTION 2.2.6 


1. (Here the indices run from 1 to n, not from 0 to n as in Eq. (6).) LOC(ALJ,K]) = 
LOC(A[0,0]) +2nJ+ 2K, where A[0,0] is an assumed node that is actually nonexistent. 
If we set J = K = 1, we get LOC(A[1,1]) = LOC(A[0,0]) + 2n + 2, so the answer can 
be expressed in several ways. The fact that LOC(A[0,0]) might be negative has led to 
many bugs in compilers and loading routines. 


2. LOC(A[T1,...,In]) = LOC(ALO,...,0]) + ycpcp@rIr = LOC(ALL,...,U.]) + 


Micr<k arIr — Vicpcp arlr, where ar = c]] 2 ,<g(us — ls + 1). 

Note: For a generalization to the structures occurring in programming languages 
such as C, and a simple algorithm to compute the relevant constants, see P. Deuel, 
CACM 9 (1966), 344-347. 


38. 1<k <j < nif and only if0 < k-1<j-—1< n—1; so replace k, j, n 
respectively by k — 1, j — 1, n — 1 in all formulas derived for lower bound zero. 


4. LOC(A[J,K]) = LOC(A[0,0]) +nJ — J(J — 1)/2 +K. 


5. Let AO = LOC(A[0,0]). There are at least two solutions, assuming that J is in 
rll and K is in rl2. (i) ‘LDA TA2,1:7’, where location TA2+j is ‘NOP j+1*j/2+A0,2’; 
(ii) ‘LDA C1,7:2’, where location C1 contains ‘NOP TA,1:7’ and location TA+j says 
‘NOP j+1*j/2+A0’. The latter takes one more cycle but doesn’t tie the table down 
to index register 2. 


6. (a) LOC(ALT,3,K]) = LOC(ALO,0,01) + za $ To +5) 
(b) LOC(B[I,J,K]) = LOC(B[0,0,0]) 
n+3 n+3-I n+2-I n+2—J 
= z K—J 
ee eg ee 
hence the stated form is possible in this case also. 


7. LOC(AIIi,...,In]) =LOC(ALO,...,01)+ $` ( 
1<r<k 
8. (Solution by P. Nash.) Let X[I,J,K] be defined for0 <I <n,0<JI<n+1, 
0<K <n+2. We can let A[I,J,K] = X[1,J,K]; B[I,J,K] = X[J,I + 1,K]; 
C[I,J,K] = X[I,K,J +1]; D[I,J,K] = X[J,K,I + 2]; E[I,J,K] = X[K,I +1,J+ 1]; 
F[I,J,K] = X[K,J+ 1,I + 2]. This scheme is the best possible, since it packs the 
(n + 1)(n + 2)(n + 3) elements of the six tetrahedral arrays into consecutive locations 
with no overlap. Proof: A and B exhaust all cells X[i, j,k] with k = min(i, j, k); 
C and D exhaust all cells with j = min(i,j,k) Æ k; E and F exhaust all cells with 
i = min(i, j, k) Æ j, k. 
(The construction generalizes to m dimensions, if anybody ever wants to pack the 
elements of m! generalized tetrahedral arrays into (n+ 1)(n +2)... (n+m) consecutive 


I,+k—r 


). See exercise 1.2.6—56. 
1+k-r 
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locations. Associate a permutation a1@2...@m with each array, and store its elements 
in X[Ia, + B1, Ias + Bo,...,1Ta,, + Bm], where B, Bz... Bm is an inversion table for 
a1@2...Q@m as defined in exercise 5.1.1-7.) 


9. G1. Set pointer variables P1, P2, P3, P4, P5, P6 to the first locations of the lists 
FEMALE, A21, A22, A23, BLOND, BLUE, respectively. Assume in what follows 
that the end of each list is given by link A, and A is smaller than any other 
link. If P6 = A, stop (the list, unfortunately, is empty). 


G2. (Many possible orderings of the following actions could be done; we have 
chosen to examine EYES first, then HAIR, then AGE, then SEX.) Set P5 + 
HAIR(P5) zero or more times until P5 < P6. If now P5 < P6, go to step G5. 


G3. Set P4 < AGE(P4) repeatedly if necessary until P4 < P6. Similarly do the 
same to P3 and P2 until P3 < P6 and P2 < P6. If now P4, P3, P2 are all 
smaller than P6, go to G5. 


G4. Set P1 < SEX(P1) until Pi < P6. If P1 = P6, we have found one of the young 
ladies desired, so output her address, P6. (Her age can be determined from 
the settings of P2, P3, and P4.) 


G5. Set P6 + EYES (P6). Now stop if P6 = A; otherwise return to G2. J 
This algorithm is interesting but not the best way to organize a list for such a search. 
10. See Section 6.5. 
11. At most 200 + 200 + 3 - 4-200 = 2800 words. 
12. VAL(QO) = c, VAL (PO) = b/a, VAL (P1) = d. 


13. It is convenient to have at the end of each list a sentinel that “compares low” in 
some field on which the list is ordered. A straight one-way list could have been used, for 
example by retaining just the LEFT links in BASEROW [i] and the UP links in BASECOL[ j], 
by modifying Algorithm S thus: In S2, test if PO = A before setting J +— COL(PO); if 
so, set PO +— LOC(BASEROW[IO]) and go to S3. In S3, test if QO = A; if so, terminate. 
Step S4 should change by analogy with step S2. In S5, test if P1 = A; if so, act as if 
COL(P1) < 0. In S6, test if UP(PTR[J]) = A; if so, act as if its ROW field were negative. 

These modifications make the algorithm more complicated and save no storage 
space except a ROW or COL field in the list heads (which in the case of MIX is no saving 
at all). 


14. One could first link together those columns that have a nonzero element in the 
pivot row, so that all other columns could be skipped as we pivot on each row. Rows 
in which the pivot column is zero are skipped over immediately. 
15. Let rll = PIVOT, J; rI2 = PO; rīI3 = QO; rl4 = P; rI5 = P1,X; LOC(BASEROW[i]) = 
BROW + i; LOC(BASECOL[j]) = BCOL + j; PTR[j] = BCOL + j (1:3). 


01 ROW EQU 0:3 

02 UP EQU 4:5 

08 COL EQU 0:3 

04 LEFT EQU 4:5 

05 PTR EQU 1:3 

06 PIVOTSTEP STJ 9F Subroutine entrance, rll = PIVOT 
07 S1 LD2 0,1(ROW) S1. Initialize. 

08 ST2 IO IO + ROW(PIVOT). 


09 LD3 1,1(COL) 
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10 ST3 JO JO + COL(PIVOT). 

11 LDA =1.0= Floating point constant 1 

12 FDIV 2,1 

13 STA ALPHA ALPHA <— 1/VAL(PIVOT). 

14 LDA =1.0= 

15 STA 2,1 VAL(PIVOT) & 1. 

16 ENT2 BROW,2 PO + LOC(BASEROW[IO]). 

17 ENT3 BCOL,3 QO + LOC(BASECOL[JO]). 

18 JMP 52 

19 2H ENTA BCOL,1 

20 STA BCOL,1(PTR) PTR[J] « LOC(BASECOL[J]). 

21 LDA 2,2 

22 FMUL ALPHA 

28 STA 2,2 VAL(PO) < ALPHA x VAL(PO). 

24 $2 LD2 1,2(LEFT) S2. Process pivot row. PO < LEFT(PO). 
25 LD1 1,2(COL) J + COL(PO). 

26 JiNN 2B If J > 0, process J. 

27 $3 LD3 0,3(UP) S3. Find new row. QO + UP (Q0). 
28 LD4 0,3(ROW) rl4 + ROW(QO). 

29 9H J4N * If rl4 < 0, exit. 

30 CMP4 IO 

31 JE 83 If rl4 = I0, repeat. 

32 ST4 I(ROW) I< r4. 

33 ENT4 BROW,4 P «+ LOC(BASEROW[I]). 

34 S4A LD5 1,4(LEFT) P1 + LEFT(P). 

35 S4 LD2 1,2(LEFT) S4. Find new column. PO + LEFT(PO). 
36 LD1 1,2(COL) J + COL(PO). 

37 CMP1 JO 

38 JE S4 Repeat if J = JO. 

39 ENTA 0,1 

40 SLA 2 rA (0:3) + J. 

41 JANN $5 

42 LDAN 2,3 If I <0, 

43 FMUL ALPHA set VAL(QO) < —ALPHA x VAL(QO). 
44 STA 2,3 

45 JMP $3 

46 1H ENT4 0,5 P & P1. 

47 LD5 1,4(LEFT) Pi + LEFT(P). 

48 S5 CMPA 1,5(COL) S5. Find I, J element. 

49 JL 1B Loop until COL(P1) < J. 

50 JE S7 If =, go right to $7. 

51 S6 LD5 BCOL,1(PTR) S6. Insert I, J element. rl5 «+ PTR[J]. 
52 LDA I rA(0:3) <I. 

58 2H ENT6 0,5 rl6 + rI5. 

54 LD5 0,6(UP) rI5 + UP(rI6). 

55 CMPA 0,5(ROW) 

56 JL 2B Jump if ROW(rI5) > I. 

57 LD5 AVAIL X < AVAIL. 

58 J5Z OVERFLOW 
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59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 S7 
72 
13 
74 
15 
16 
17 
18 
79 S8 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
91 
92 


LDA 
STA 
LDA 
STA 
LDA 
STA 
ST1 
LDA 
STA 
STZ 
STS 
STS 
LDAN 
FMUL 
FADD 
JAZ 
STA 
STS 
ENT4 
JMP 
LD6 
JMP 
LD6 
LDA 
DECA 
JANZ 
LDA 
STA 
LDA 
STA 
LDA 
STA 
ST5 
JMP 


0,5(UP) 
AVAIL 
0,6(UP) 
0,5(UP) 
1,4 (LEFT) 
1,5 (LEFT) 
1,5(COL) 
I (RoW) 
0,5(ROW) 
2,5 

1,4 (LEFT) 
0,6(UP) 
2,3 

2,2 

2,5 

s8 

2,5 
BCOL,1(PTR) 
0,5 

S4A 
BCOL, 1 (PTR) 
*+2 
0,6(UP) 
0,6(UP) 
0,5 

*-3 
0,5(UP) 
0,6(UP) 
1,5 (LEFT) 
1,4 (LEFT) 
AVAIL 
0,5(UP) 
AVAIL 
S4A 
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UP(X) 4+ UP(PTR[J]). 


LEFT(X) + LEFT(P). 
COL(X) + J. 


ROW(X) + I. 
VAL(X) + 0. 
LEFT(P) + X. 
UP(PTR[J]) + X. 
S7. Pivot. —VAL(QO) 

x VAL(PO) 

+ VAL (P1). 

If significance lost, to S8. 
Otherwise store in VAL (P1). 
PTR[J] + P1. 
P< P1. 
P1 4+ LEFT (P), to S4. 
S8. Delete I, J element. rl6 + PTR[J]. 


rl6 + UP(rI6). 


Is UP(rI6) = P1? 
Loop until equal. 


UP(rI6) «+ UP (P1). 
LEFT(P) + LEFT(P1). 


AVAIL = P1. 


P1 + LEFT(P), to S4. I 
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Note: Using the conventions of Chapter 4, lines 71-74 would actually be coded 


LDA 2,3; FMUL 2,2; FCMP 2,5; JE S8; STA TEMP; LDA 2,5; FSUB TEMP; 


with a suitable parameter EPSILON in location zero. 


17. For each row 7 and each element A[i,k] 40, add A[i,k] times row k of B to row i 
of C. Maintain only the COL links of C while doing this; the ROW links are easily filled in 
afterwards. [A. Schoor, Inf. Proc. Letters 15 (1982), 87-89.] 


18. The three pivot steps, in respective columns 3, 1, 2, yield respectively 


Ole WI w= 


WIN Wl wily 


Wl WIN w= 


4 20 0 10 
= Pile |= : 
-4 -3 0 1 -2 0 
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after the final permutations, we have the answer 


1 —2 1 
0 hk -=2 
0 0 1 


20. ao = LOC(A[1,1]) — 3, a1 = 1 or 2, a2 = 3 — a1. 


21. For example, M + max(I,J), LOC(A[I,J]) = Loc(A[1,1]) +M(M — 1)+ I-J. 
(Such formulas have been proposed independently by many people. A. L. Rosen- 
berg and H. R. Strong have suggested the following k-dimensional generalization: 
LOC(A[I1,...,Ig]) = Lẹ where Lı = LOC(ALI,...,1]) +11—1, L, =L,-1+(M,—1)"4 
(M. — I )(MT} — (M, —1)"~'), and M, = max(Ti,...,1,-) [IBM Technical Disclosure 
Bulletin 14 (1972), 3026-3028]. See Current Trends in Programming Methodology 4 
(Prentice-Hall, 1978), 263-311, for further results of this kind.) 


22. According to the combinatorial number system (exercise 1.2.6—56), we can let 


pisaia (4 (ATR) gg (Et DY 


[Det Kongelige Norske Videnskabers Selskabs Forhandlinger 34 (1961), 8-9.] 


23. Let c[J] = LOC(A[0,J]) = LOC(A[0,0])+mJ, if there were m rows when the matrix 
grew from J to J+ 1 columns; similarly, let r[I] = LOC(A[I,0]) = LOC(A[0,0]) +nI, if 
there were n columns when we created row I. Then we can use the allocation function 
T+], if [J] > v(t; 

J+ r[I], otherwise. 


LOC(A[I,J]) = { 


It is not hard to prove that c[J] > r[I] implies c[J] > r[I] + J, and c[J] < r[I] implies 
c[J] + I < r[I]; therefore the relation 


LOC(ALI, J]) = max(I + LOC(A[0, J]),J + LOC(A[I,0])) 


also holds. We need not restrict allocation to mn consecutive locations; the only 
constraint is that, when the matrix grows, we allocate m or n consecutive new cells in 
locations greater than those previously used. This construction is due to E. J. Otoo 
and T. H. Merrett [Computing 31 (1983), 1-9], who also generalized it to k dimensions. 


24. {Aho, Hopcroft, and Ullman, The Design and Analysis of Computer Algorithms 
(Addison—Wesley, 1974), exercise 2.12.] Besides the array A, maintain also a verification 
array V of the same size, and a list L of the locations used. Let n be the number of 
items in L; initially n = 0 and the contents of L, A, and V are arbitrary. Whenever you 
want to access A[k] for a value of k that you might not have used before, first check 
whether 0 < V[k] < n and LIV[k]] = k. If not, set Vik] «+ n, Lin] + k, A[k] <0, 
and n + n+1. Otherwise you can be sure that A[k] already contains legitimate data. 
(By a slight extension of this method, it is possible to save and eventually restore the 
contents of all entries of A and V that change during the computation.) 


SECTION 2.3 


1. There are three ways to choose the root. Once the root has been chosen, say A, 
there are three ways to partition the other nodes into subtrees: {B}, {C}; {C}, {B}; 
{B,C}. In the latter case there are two ways to make {B,C} into a tree, depending 
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on which is the root. Hence we get A A 
the four trees shown when A is the (A) Q @) @ 
root, and 12 in all. This problem is 

solved for any number n of nodes ©) © © @) ®) © 


in exercise 2.3.4.4—23. 
© © 


2. The first two trees in the answer to exercise 1 are the same, as oriented trees, so 
we get only 9 different possibilities in this case. For the general solution, see Section 
2.3.4.4, where the formula n”~* is proved. 


3. Part 1: To show there is at least one such sequence. Let the tree have n nodes. The 
result is clear when n = 1, since X must be the root. If n > 1, the definition implies 
there is a root Xı and subtrees Ti, T2,..., Tm; either X = Xı or X is a member of a 
unique T}. In the latter case, there is by induction a path X2,...,X where Xə is the 
root of Tj, and since X1 is the parent of Xə we have a path X1, X2,..., X. 

Part 2: To show there is at most one such sequence. We will prove by induction 
that if X is not the root of the tree, X has a unique parent (so that Xp determines X;,_-1 
determines X;~2, etc.) If the tree has one node, there is nothing to prove; otherwise 
X is in a unique Tj. Either X is the root of T}, in which case X has a unique parent 
by definition; or X is not the root of Tj, in which case X has a unique parent in T; by 
induction, and no node outside of Tj can be X’s parent. 


4. True (unfortunately). 
5. 4. 


6. Let parent!!(X) denote X, and let parent t! (xX) = parent(parent!*!(X)), so 
that parent!) (X) is X’s parent, and parent?! (X) is X’s grandparent; when k > 2, 
parent" (X) is X’s “(great-)*~*grandparent.” The requested cousinship condition is 
that parent” t! (Xx) = parent” +”+1(Y) but parent™! (X) 4 parent!™*"l(Y). When 
n > 0, this relation is not symmetrical with respect to X and Y, although people 
usually treat it as symmetrical in everyday conversation. 


7. Use the (unsymmetric) condition defined in exercise 6, with the convention that 
parent’! (X) 4 parent” (Y) if either j or k (or both) is —1. To show that this relation 
is always valid for some unique m and n, consider the Dewey decimal notation for X 
and Y, namely 1.a1.---.dp.b1.---.bg and 1.ay.--+.dp.c1.-+-+.cr, where p > 0, q => 0, 
r > 0 and (if gr £ 0) bı # cı. The Dewey numbers of any pair of nodes can be written 
in this form, and clearly we must take m = q—1andm+n=r-—l. 


8. No binary tree is really a tree; the concepts are quite separate, even though the 
diagram of a nonempty binary tree may look treelike. 


9. A is the root, since we conventionally put the root at the top. 


10. Any finite collection of nested sets corresponds to a forest as defined in the text, as 
follows: Let Ai,..., An be the sets of the collection that are contained in no other. For 
fixed j, the sub-collection of all sets contained in A; is nested; hence we may assume 
that this sub-collection corresponds to a tree (unordered) with A; as the root. 


11. In a nested collection C let X = Y if there is some Z € C such that XUY C Z. 
This relation is obviously reflexive and symmetric, and it is in fact an equivalence 
relation since W = X and X = Y implies that there are Zı and Z2 in C with W C 7%, 
X C ZN Z2, and Y C Zə. Since Zı N Z2 Æ Y, either Zı C Z2 or Z2 C Zi; hence 
WUY C ZU Z2 € C. Now if C is a nested collection, define an oriented forest 
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corresponding to C by the rule “X is an ancestor of Y, and Y is a descendant of X 
(that is, a proper ancestor or descendant), if and only if X D Y.” Each equivalence class 
of C corresponds to an oriented tree, which is an oriented forest with X = Y for all X, 
Y. (We thereby have generalized the definitions of forest and tree that were given for 
finite collections.) In these terms, we may define the level of X as the cardinal number 
of ancestors(X). Similarly, the degree of X is the cardinal number of equivalence classes 
in the nested collection descendants(X). We say X is the parent of Y, and Y is a child 
of X, if X is an ancestor of Y but there is no Z such that X D Z D Y. (It is possible for 
X to have descendants but no children, ancestors but no parent.) To get ordered trees 
and forests, order the equivalence classes mentioned above in some ad hoc manner, for 
example by embedding the relation C into linear order as in exercise 2.2.3-14. 

Example (a): Let Sak = {x | x = .dided3... in decimal notation, where a = 
.e1e2e3... in decimal notation, and dj = ej if j mod2" 4 0}. The collection C = 
{Sak |k => 0,0 < a < 1} is nested, and gives a tree with infinitely many levels and 
uncountable degree for each node. 

Example (b), (c): It is convenient to define this set in the plane, instead of in terms 
of real numbers, and this is sufficient since there is a one-to-one correspondence between 
the plane and the real numbers. Let Samn = {(a,y) | m/2" < y < (m+ 1)/2"}, 
and let Ta = {(x,y) | £x < a}. The collection C = {Samn | 0 < a < 1,n > 0, 
0 < m < 2°} U{T. | 0 < a < 1} is easily seen to be nested. The children of 
Samn are Sa(2m)(n41) and Sa(2m+1)(n+1); and Ta has the child Saoo plus the subtree 
{Sgmn | B < a} U {Tg | 8 < a}. So each node has degree 2, and each node has 
uncountably many ancestors of the form Ta. This construction is due to R. Bigelow. 

Note: If we take a suitable well-ordering of the real numbers, and if we define 
Ta = {(x,y) | £ > a}, we can improve this construction slightly, obtaining a nested 
collection where each node has uncountable level, degree 2, and two children. 


12. We impose an additional condition on the partial ordering (analogous to that of 
“nested sets”) to ensure that it corresponds to a forest: If x < y and x < z then either 
y x zor zx y. In other words, the elements larger than any given element are linearly 
ordered. To make a tree, also assert the existence of a largest element r such that x < r 
for all x. A proof that this gives an unordered tree as defined in the text, when the 
number of nodes is finite, runs like the proof for nested sets in exercise 10. 


13. a@1.a2.-+++.dp, Q1-Q2.***.Qk—1; ---, G1-G2, Q1. 


14. Since S is nonempty, it contains an element 1.a1.---.a, where k is as small as 
possible; if k > 0 we also take a, as small as possible in S, and we immediately see that 
k must be 0. In other words, S must contain the element 1. Let 1 be the root. All other 
elements have k > 0, and so the remaining elements of S can be partitioned into sets 
S; = {l.j.a2.---.ax}, 1 < j < m, for some m > 0. If m 4 0 and Sm is nonempty, we 
deduce by reasoning as above that 1.j is in S} for each S;; hence each S; is nonempty. 
Then it is easy to see that the sets S; = {1.a2.---.ak | 1.j.a2. -++ .ap is in Sj} satisfy 
the same condition as S did. By induction, each of the S; forms a tree. 


15. Let the root be 1, and let the roots of the left and right subtrees of a be a.0 
and a.1, respectively, when such roots exist. For example, King Christian IX appears 
in two positions of Fig. 18(a), namely 1.0.0.0.0 and 1.1.0.0.1.0. For brevity we may 
drop the decimal points and write merely 10000 and 110010. Note: This notation is 
due to Francis Galton; see Natural Inheritance (Macmillan, 1889), 249. For pedigrees, 
it is more mnemonic to use F and M in place of 0 and 1 and to drop the initial 1; 
thus Christian IX is Charles’s MF FMF, his mother’s father’s father’s mother’s father. 
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The 0 and 1 convention is interesting for another reason: It gives us an important 
correspondence between nodes in a binary tree and positive integers expressed in the 
binary system (namely, memory addresses in a computer). 


16. (a) (x) (b) © or 
O Q OO OW GORN OR 
O 0000 OO © & 


A; parent (Z[1,2]) = C; parent(Z[1, 2,2]) = 


2). L[3,1] is nonsense, since L[3] is an empty List. 


17. parent(Z[1]) = 

18. L{5,1,1] =( 

19. * [L] L[2] = (L); L[2,1,1] = a. 
~~ 


20. (Intuitively, the correspondence between 0-2-trees and binary trees is obtained by 
removing all terminal nodes of the 0-2-tree; see the important construction in Section 
2.3.4.5.) Let a 0-2-tree with one node correspond to the empty binary tree; and let 
a 0-2-tree with more than one node, consisting therefore of a root r and 0-2-trees Ti 
and T2, correspond to the binary tree with root r, left subtree T}, and right subtree T3, 
where T; and Tz correspond respectively to Ti and T3. 


21. 14+0-ny+1-no+---+(m—1)-nm. Proof: The number of nodes in the tree 
is no + nı + n2 +--+ Nm, and this also equals 1 + (number of children in the tree) 
=14+0-no+1-n14+2-ne+---+m-nm. 


22. The basic idea is to proceed recursively, with the representation of a nonempty 
binary tree defined to be the representation of its root plus half-size-and-rotated rep- 
resentations of its left and right subtrees. Thus an arbitrarily large binary tree can be 
represented on a single sheet of paper, if one has a sufficiently powerful magnifying glass. 

Many variations on this theme are possible. For example, one idea is to represent 
the root by a line from the center of a given landscape-oriented page to the top edge, 
and to rotate the left-subtree representation by 90° clockwise in the left halfpage, the 
right-subtree representation by 90° counterclockwise in the right halfpage. Each node 
is then represented by a line. (When this method is applied to a complete binary tree 
having 2” —1 nodes on k levels, it yields so-called “H-trees,” which are the most efficient 
layouts of such binary trees on a VLSI chip; see R. P. Brent and H. T. Kung, Inf. Proc. 
Letters 11 (1980), 46—48.) 


J H 
A oe H F J 
D F 
B A Cc 
B Cc 
E 
=== D E G 
G 
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Another idea is to represent an empty binary tree by some sort of box, and to rotate the 
subtree representations of nonempty binary trees so that left subsubtrees are alternately 
to the left of or below the corresponding right subsubtrees, depending on whether the 
depth of recursion is even or odd. Then the boxes correspond to external nodes in 
an extended binary tree (see Section 2.3.4.5). This representation, which is strongly 
related to the 2-d trees and quadtrees discussed in Section 6.5, is especially appropriate 
when the external nodes carry information but the internal nodes do not. 


SECTION 2.3.1 
1. INFO(T) = A, INFO(RLINK(T)) = C, etc.; the answer is H. 
2. Preorder: 1245367; symmetric order: 4251637; postorder: 4526731. 


3. The statement is true; notice, for example, that nodes 4, 5, 6, 7 always appear in 
this order in exercise 2. The result is immediately proved by induction on the size of 
the binary tree. 


4. It is the reverse of postorder. (This is easily proved by induction.) 


5. In the tree of exercise 2, for example, preorder is 1, 10, 100, 101, 11, 110, 111, using 
binary notation (which is in this case equivalent to the Dewey system). The strings of 
digits have been sorted, like words in a dictionary. 

In general, the nodes will be listed in preorder if they are sorted lexicographically 
from left to right, with “blank” < 0 < 1. The nodes will be listed in postorder if they 
are sorted lexicographically with 0 < 1 < “blank”. For inorder, use 0 < “blank” < 1. 

(Moreover, if we imagine the blanks at the left and treat the Dewey labels as 
ordinary binary numbers, we get level order; see 2.3.3-(8).) 


6. The fact that pip2...pn is obtainable with a stack is readily proved by induction 
on n, or in fact we may observe that Algorithm T does precisely what is required in 
its stack actions. (The corresponding sequence of S’s and X’s, as in exercise 2.2.1—3, is 
the same as the sequence of 1s and 2s as subscripts in double order; see exercise 18.) 

Conversely, if pı p2...pn is obtainable with a stack and if pẹ = 1, then pi... Pk—1 is 
a permutation of {2,...,k} and pr41...DPn is a permutation of {k + 1,..., n}; these are 
the permutations corresponding to the left and right subtrees, and both are obtainable 
with a stack. The proof now proceeds by induction. 


7. From the preorder, the root is known; then from the inorder, we know the left 
subtree and the right subtree; and in fact we know the preorder and inorder of the 
nodes in the latter subtrees. Hence the tree is readily constructed (and indeed it 
is quite amusing to construct a simple algorithm that links the tree together in the 
normal fashion, starting with the nodes linked together in preorder in LLINK and in 
inorder in RLINK). Similarly, postorder and inorder together characterize the structure. 
But preorder and postorder do not; there are two binary trees having AB as preorder 
and BA as postorder. If all nonterminal nodes of a binary tree have both branches 
nonempty, its structure is characterized by preorder and postorder. 


8. (a) Binary trees with all LLINKs null. (b) Binary trees with zero or one nodes. 
(c) Binary trees with all RLINKs null. 

9. T1 once, T2 2n+1 times, T3 n times, T4 n+1 times, T5 n times. These counts 
can be derived by induction or by Kirchhoff’s law, or by examining Program T. 


10. A binary tree with all RLINKs null will cause all n node addresses to be put in the 
stack before any are removed. 
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11. Let aną be the number of binary trees with n nodes for which the stack in 
Algorithm T never contains more than k items. If gx(z) = }_„ ankz”, we find gi(z) = 
1/(1=2), ga(z) = 1/(1-2/(1-2)) = (1 = 2)/(1= 22), .-., ge(2) = 1/(1 — zgr-1(2)) = 
qk-1(2)/qk(z) where q-1(z) = qo(2) = 1, qk+1(2) = qk(z) — 2qu-1(2); hence gp(z) = 
(fa(z)*t* — fo(z)***)/(fu(z)**? — fo(z)**?) where f;(z) = 4(1 + VI—4z). It can 
now be shown that ang = [u"] (1 — u)(1 + u)?"(1 — u*®*1)/(1 — u*t?); hence sn = 
Ygs klank — Gn(e—1)) is [ut] (1 — u)?(1 + u)?” ee u/(1 — uf), minus ann. The 
technique of exercise 5.2.2-52 now yields the asymptotic series 


_ 3 13 m 1- -3/2 
Sn/Ann = Vrn 2 at rea O(n ): 
[N. G. de Bruijn, D. E. Knuth, and S. O. Rice, in Graph Theory and Computing, ed. 
by R. C. Read (New York: Academic Press, 1972), 15-22.] 

When the binary tree represents a forest as described in Section 2.3.2, the quantity 
analyzed here is the height of that forest (the furthest distance between a node and a 
root, plus one). Generalizations to many other varieties of trees have been obtained by 
Flajolet and Odlyzko [J. Computer and System Sci. 25 (1982), 171-213]; the asymptotic 
distribution of heights, both near the mean and far away, was subsequently analyzed 
by Flajolet, Gao, Odlyzko, and Richmond [Combinatorics, Probability, and Computing 
2 (1993), 145-156]. 

12. Visit NODE(P) between steps T2 and T3, instead of in step T5. For the proof, 
demonstrate the validity of the statement “Starting at step T2 with ... original value 
A[1] ...A[m],” essentially as in the text. 

13. (Solution by S. Aratijo, 1976.) Let steps T1 through T4 be unchanged, except that 
a new variable Q is initialized to A in step T1’; Q will point to the last node visited, if 
any. Step T5 becomes two steps: 

T5’. [Right branch done?] If RLINK(P) = A or RLINK(P) = Q, go on to T6; 

otherwise set A <= P, P + RLINK(P) and return to T2’. 

T6’. [Visit P.] Visit NODE(P), set Q + P, and return to T4’. 

A similar proof applies. (Steps T4’ and T5’ can be streamlined so that nodes are not 
taken off the stack and immediately reinserted.) 


14. By induction, there are always exactly n+ 1 A links (counting T when it is null). 
There are n nonnull links, counting T, so the remark in the text about the majority of 
null links is justified. 

15. There is a thread LLINK or RLINK pointing to a node if and only if it has a nonempty 
right or left subtree, respectively. (See Fig. 24.) 

16. If LTAG(Q) = 0, Q»* is LLINK (Q); thus Q» is one step down and to the left. Otherwise 
Q* is obtained by going upwards in the tree (if necessary) repeatedly until the first time 
it is possible to go down to the right without retracing steps; typical examples are the 
trips from P to Px and from Q to Q» in the following tree: 


566 ANSWERS TO EXERCISES 2.3.1 


17. If LTAG(P) = 0, set Q + LLINK(P) and terminate. Otherwise set Q «+ P, then set 
Q + RLINK(Q) zero or more times until finding RTAG(Q) = 0; finally set Q 4 RLINK(Q) 
once more. 


18. Modify Algorithm T by inserting a step T2.5, “Visit NODE(P) the first time”; in 


step T5, we are visiting NODE(P) the second time. 
Given a threaded tree the traversal is extremely simple: 


(P, 1)“ = (LLINK(P), 1) if LTAG(P) = 0, otherwise (P, 2); 
(P, 2)“ = (RLINK(P), 1) if RTAG(P) = 0, otherwise (RLINK (P), 2). 


In each case, we move at most one step in the tree; in practice, therefore, double order 
and the values of d and e are embedded in a program and not explicitly mentioned. 

Suppressing all the first visits gives us precisely Algorithms T and S; suppressing 
all the second visits gives us the solutions to exercises 12 and 17. 


19. The basic idea is to start by finding the parent Q of P. Then if P A LLINK(Q) we 
have Pt = Q; otherwise we can find Pi by repeatedly setting Q < Qs zero or more times 
until RTAG(Q) = 1. (See, for example, P and Pi in the tree shown.) 


There is no efficient algorithm to find the parent of P in a general right-threaded 
tree, since a degenerate right-threaded tree in which all left links are null is essentially a 
circular list in which the links go the wrong way. Therefore we cannot traverse a right- 
threaded tree in postorder with the same efficiency as the stack method of exercise 13, 
if we keep no history of how we have reached the current node P. 

But if the tree is threaded in both directions, we can find P’s parent efficiently: 


F1. Set Q + P and R & P. 
F2. If LTAG(Q) = RTAG(R) = 0, set Q + LLINK(Q) and R + RLINK(R) and repeat 
this step. Otherwise go to F4 if RTAG(R) = 1. 


F3. Set Q < LLINK(Q), and terminate if P = RLINK(Q). Otherwise set R + 
RLINK(R) zero or more times until RTAG(R) = 1, then set Q < RLINK(R) and 
terminate. 


F4. Set R + RLINK(R), and terminate with Q + R if P = LLINK(R). Otherwise set 
Q «+ LLINK(Q) zero or more times until LTAG(Q) = 1, then set Q < LLINK(Q) 
and terminate. J 


The average running time of Algorithm F is O(1) when P is a random node of the tree. 
For if we count only the steps Q + LLINK(Q) when P is a right child, or only the steps 
R < RLINK(R) when P is a left child, each link is traversed for exactly one node P. 
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20. Replace lines 06-09 by: Replace lines 12-13 by: 
T3 ENT4 0,6 LD4 0,6(LINK) 
LD6 AVAIL LD5 0,6(INF0) 
J6Z OVERFLOW LDX AVAIL 
LDX  0,6(LINK) STX  0,6(LINK) 
STX AVAIL ST6 AVAIL 
ST5 0,6(INFO) ENT6 0,4 


ST4  0,6(LINK) 
If two more lines of code are added at line 06 


T3 LD3 0,5(LLINK) 
J3Z T5 To T5 if LLINK (P) = A. 


with appropriate changes in lines 10 and 11, the running time goes down from (30n + 
a + 4)u to (27a + 6n — 22)u. (This same device would reduce the running time of 
Program T to (12a + 6n — 7)u, which is a slight improvement, if we set a = (n+ 1)/2.) 
21. The following solution by Joseph M. Morris [Inf. Proc. Letters 9 (1979), 197-200] 
traverses also in preorder (see exercise 18). 

U1. [Initialize] Set P+ T and R + A. 

U2. [Done?] If P = A, the algorithm terminates. 

U3. [Look left.] Set Q + LLINK(P). If Q = A, visit NODE(P) in preorder and go 
to U6. 

UA. [Search for thread.] Set Q + RLINK(Q) zero or more times until either Q = R 
or RLINK(Q) = A. 

U5. [Insert or remove thread.] If Q Æ R, set RLINK(Q) + P and go to U8. Otherwise 
set RLINK(Q) + A (it had been changed temporarily to P, but we’ve now 
traversed P’s left subtree). 

6. [Inorder visit.] Visit NODE (P) in inorder. 

7. [Go to right or up.] Set R + P, P + RLINK(P), and return to U2. 

8. [Preorder visit.] Visit NODE(P) in preorder. 

9. [Go to left.] Set P + LLINK(P) and return to step U3. J 

Morris also suggested a slightly more complicated way to traverse in postorder. 

A completely different solution was found by J. M. Robson [Inf. Proc. Letters 2 
(1973), 12-14]. Let’s say that a node is “full” if its LLINK and RLINK are nonnull, 
“empty” if its LLINK and RLINK are both empty. Robson found a way to maintain a 
stack of pointers to the full nodes whose right subtrees are being visited, using the link 
fields in empty nodes! 

Yet another way to avoid an auxiliary stack was discovered independently by G. 
Lindstrom and B. Dwyer, Inf. Proc. Letters 2 (1973), 47-51, 143-145. Their algorithm 
traverses in triple order—it visits every node exactly three times, once in each of 
preorder, inorder, and postorder — but it does not know which of the three is currently 
being done. 


W1. [Initialize] Set P + T and Q + S, where S is a sentinel value—any number 
that is known to be different from any link in the tree (e.g., —1). 


W2. [Bypass null.] If P = A, set P + Q and Q¢ A. 


W3. [Done?] If P = S, terminate the algorithm. (We will have Q = T at termina- 
tion.) 
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W4. [Visit.] Visit NODE (P). 


W5. [Rotate.] Set R 4+ LLINK(P), LLINK(P) + RLINK(P), RLINK(P) + Q, Q + P, 
P + R, and return to W2. J 


Correctness follows from the fact that if we start at W2 with P pointing to the root of 
a binary tree T and Q pointing to X, where X is not a link in that tree, the algorithm 
will traverse the tree in triple order and reach step W3 with P = X and Q =T. 

If a(T) = £1£2...£3n is the resulting sequence of nodes in triple order, we have 
a(T) = T a(LLINK(T)) T a(RLINK(T)) T. Therefore, as Lindstrom observed, the three 
subsequences %1%4...%3n—2, T275 ...L3n—1; T3T6 ...L3n each include every tree node 
just once. (Since x;+1 is either the parent or child of zj, these subsequences visit the 
nodes in such a way that each is at most three links away from its predecessor. Section 
7.2.1.6 describes a general traversal scheme called prepostorder that has this property 
not only for binary trees but for trees in general.) 


22. This program uses the conventions of Programs T and S, with Q in rI6 and/or rI4. 
The old-fashioned MIX computer is not good at comparing index registers for equality, 
so variable R is omitted and the test “Q = R” is changed to “RLINK(Q) = P”. 


01 Ui LD5 HEAD(LLINK) 1 U1. Initialize. P 4+ T. 

02 U2A J5Z DONE 1 Stop if P = A. 

03 U3 LD6 0,5(LLINK) n+a-1 U3. Look left. Q + LLINK (P). 

04 J6Zz U6 n+a-1 To U6 if Q = A. 

05 U4 CMP5 1,6(RLINK) 2n—2b U4. Search for thread. 

06 JE 5F 2n— 2b Jump if RLINK(Q) = P. 

07 ENT4 0,6 2n—2b—a+1 rl4<¢Q. 

08 LD6 1,6(RLINK) 2n—2b—a+1 

09 J6NZ U4 2n—2b—a+1 To U4 with Q + RLINK (Q) if it’s $ 0. 
10 U5 STS 1,4(RLINK) a—1 U5a. Insert thread. RLINK (Q) < P. 
11 U9 LD5 0,5(LLINK) a—1 U9. Go to left. P + LLINK(P). 

12 JMP U3 a—1 To U3. 

13 5H STZ 1,6(RLINK) a—1 U5b. Remove thread. RLINK(Q) + A. 
14 U6 JMP VISIT n U6. Inorder visit. 

15 U7 LDS 1,5(RLINK) n U7. Go to right or up. P < RLINK(P). 
16 U2 J5NZ U3 n U2. Done? To U3 if PF A. 

17 DONE ... I 


The total running time is 21n + 6a — 3 — 14b, where n is the number of nodes, a is 
the number of null RLINKs (hence a — 1 is the number of nonnull LLINKs), and b is the 
number of nodes on the tree’s “right spine” T, RLINK (T), RLINK(RLINK(T)), etc. 


23. Insertion to the right: RLINKT(Q) + RLINKT(P), RLINK(P) + Q, RTAG(P) + 0, 
LLINK(Q) + A. Insertion to the left, assuming LLINK(P) = A: Set LLINK(P) + Q, 
LLINK(Q) + A, RLINK(Q) + P, RTAG(Q) + 1. Insertion to the left, between P and 
LLINK(P) Æ A: Set R 4+ LLINK(P), LLINK(Q) + R, and then set R + RLINK(R) zero or 
more times until RTAG(R) = 1; finally set RLINK(R) «+ Q, LLINK(P) + Q, RLINK(Q) +P, 
RTAG(Q) + 1. 

(A more efficient algorithm for the last case can be used if we know a node F 
such that P = LLINK(F) or P = RLINK(F); assuming the latter, for example, we could 
set INFO(P) <+ INFO(Q), RLINK(F) + Q, LLINK(Q) + P, RLINKT(Q) + RLINKT(P), 
RLINK(P) «+ Q, RTAG(P) + 1. This takes a fixed amount of time, but it is generally not 
recommended because it switches nodes around in memory.) 
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24. No: 


25. We first prove (b), by induction on the number of nodes in T, and similarly (c). 
Now (a) breaks into several cases; write T <1 T” if (i) holds, T <2 T’ if (ii) holds, etc. 
Then T <, T’ and T’ < T” implies T <; T”; T <2 T’ and T’ < T” implies T <2 T”; 
and the remaining two cases are treated by proving (a) by induction on the number of 
nodes in T. 


26. If the double order of T is (ui, d1), (u2, d2), . . . , (U2n, dan) where the u’s are nodes 
and the d’s are 1 or 2, form the “trace” of the tree (v1, $1), (v2, $2),---, (Van, S2n ), where 
vj = info(u;), and sj = l(uj) or r(u;) according as dj = 1 or 2. Now TXT" if and only 
if the trace of T (as defined here) lexicographically precedes or equals the trace of T’. 
Formally, this means that we have either n < n’ and (v,;,8;) = (vj, s4) for 1 < j < 2n, 
or else there is a k for which (v;,8;) = (vj, s4) for 1 < j < k and either vp < vp or 
Up = Vz, and Sp < Sh. 

27. R1. [Initialize.] Set P + HEAD, P’ + HEAD’; these are the respective list heads of 
the given right-threaded binary trees. Go to R3. 


R2. [Check INFO.) If INFO(P) < INFO(P’), terminate (T < T’); if INFO(P) > 
INFO(P’), terminate (T > T’). 


R3. [Go to left.] If LLINK(P) = A = LLINK(P’), go to R4; if LLINK(P) = A # 
LLINK(P’), terminate (T < T’); if LLINK(P) #4 A = LLINK(P’), terminate 
T > T’); otherwise set P + LLINK(P), P’ + LLINK(P’), and go to R2. 


R4. [End of tree?] If P = HEAD (or, equivalently, if P’ = HEAD’), terminate 
T is equivalent to T’). 


R5. [Go to right.] If RTAG(P) = 1 = RTAG(P’), set P + RLINK (P), P’ + RLINK(P’), 
and go to R4. If RTAG(P) = 1 Æ RTAG(P’), terminate (T < T’). If RTAG(P) # 
1 = RTAG(P’), terminate (T > T’). Otherwise, set P +— RLINK(P), P’ © 
RLINK(P’), and goto R2. J 


To prove the validity of this algorithm (and therefore to understand how it works), 
one may show by induction on the size of the tree Tọ that the following statement is 
valid: Starting at step R2 with P and P’ pointing to the roots of two nonempty right- 
threaded binary trees To and Tj, the algorithm will terminate if To and Tj are not 
equivalent, indicating whether To < Tj or To > To; the algorithm will reach step R4 
if To and Tj are equivalent, with P and P’ then pointing respectively to the successor 
nodes of To and Tj in symmetric order. 


28. Equivalent and similar. 


29. Prove by induction on the size of T that the following statement is valid: Starting 
at step C2 with P pointing to the root of a nonempty binary tree T and with Q pointing 
to a node that has empty left and right subtrees, the procedure will ultimately arrive 
at step C6 after setting INFO(Q) <+ INFO(P) and attaching copies of the left and right 
subtrees of NODE(P) to NODE(Q), and with P and Q pointing respectively to the preorder 
successor nodes of the trees T and NODE(Q). 
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30. Assume that the pointer T in (2) is LLINK(HEAD) in (10). Thus LOC(T) = HEAD, 
and HEADS is the first node of the binary tree in symmetric order. 
L1. [Initialize.]} Set Q<- HEAD, RLINK(Q) + Q. 
L2. [Advance.] Set P + Qs. (See below.) 
L3. [Thread.] If RLINK(Q) = A, set RLINK(Q) + P, RTAG(Q) + 1; otherwise set 
RTAG(Q) + 0. If LLINK(P) = A, set LLINK(P) + Q, LTAG(P) + 1; otherwise 
set LTAG(P) + 0. 
L4. [Done?] If P 4 HEAD, set Q + P and return to L2. J 
Step L2 of this algorithm implies the activation of an inorder traversal coroutine 
like Algorithm T, with the additional proviso that Algorithm T visits HEAD after it has 
fully traversed the tree. This notation is a convenient simplification in the description of 
tree algorithms, since we need not repeat the stack mechanisms of Algorithm T over and 
over again. Of course Algorithm S cannot be used during step L2, since the tree hasn’t 
been threaded yet. But Algorithm U in answer 21 can be used in step L2; it provides 
us with a very pretty method that threads a tree without using any auxiliary stack. 
31. X1. Set P < HEAD. 
X2. Set Q + Ps (using, say, Algorithm S, modified for a right-threaded tree). 
X3. If P Æ HEAD, set AVAIL <P. 
X4. If Q Æ HEAD, set P + Q and go back to X2. 
X5. Set LLINK(HEAD) + A. J 
Other solutions that decrease the length of the inner loop are clearly possible, although 
the order of the basic steps is somewhat critical. The stated procedure works because 
we never return a node to available storage until after Algorithm S has looked at both 
its LLINK and its RLINK; as observed in the text, each of these links is used precisely 
once during a complete tree traversal. 
32. RLINK(Q) + RLINK (P), SUC(Q) + SUC(P), SUC(P) + RLINK (P) + Q, PRED (Q) + P, 
PRED (SUC (Q)) + Q. 
33. Inserting NODE(Q) just to the left and below NODE (P) is quite simple: Set LLINKT(Q) 
<4— LLINKT(P), LLINK(P) + Q, LTAG(P) + 0, RLINK(Q) + A. Insertion to the right is 
considerably harder, since it essentially requires finding *Q, which is of comparable 
difficulty to finding Që (see exercise 19); the node-moving technique discussed in ex- 
ercise 23 could perhaps be used. So general insertions are more difficult with this 
type of threading. But the insertions required by Algorithm C are not as difficult as 
insertions are in general, and in fact the copying process is slightly faster for this kind 
of threading: 
C1. Set P + HEAD, Q + U, go to C4. (The assumptions and philosophy of 
Algorithm C in the text are being used throughout.) 
C2. If RLINK(P) # A, set R < AVAIL, LLINK(R) < LLINK(Q), LTAG(R) + 1, 
RLINK(R) «+ A, RLINK(Q) + LLINK(Q) & R. 
C3. Set INFO(Q) + INF0(P). 
C4. If LTAG(P) = 0, set R < AVAIL, LLINK(R) + LLINK(Q), LTAG(R) < 1, 
RLINK(R) + A, LLINK(Q) + R, LTAG(Q) «+ 0. 
C5. Set P + LLINK(P), Q + LLINK(Q). 
C6. If P AHEAD, goto C2. I 


The algorithm now seems almost too simple to be correct! 
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Algorithm C for threaded or right-threaded binary trees takes slightly longer due 
to the extra time to calculate Px, Q* in step C5. 

It would be possible to thread RLINKs in the usual way or to put #P in RLINK(P), in 
conjunction with this copying method, by appropriately setting the values of RLINK(R) 
and RLINKT(Q) in steps C2 and C4. 


34. A1. Set Q + P, and then repeatedly set Q + RLINK(Q) zero or more times until 
RTAG(Q) = 1. 


A2. Set R «+ RLINK(Q). If LLINK(R) = P, set LLINK(R) +- A. Otherwise set 
R «+ LLINK(R), then repeatedly set R + RLINK(R) zero or more times until 
RLINK(R) = P; then finally set RLINKT(R) < RLINKT(Q). (This step has 
removed NODE(P) and its subtrees from the original tree.) 


A3. Set RLINK(Q) + HEAD, LLINK (HEAD) <P. JJ 


(The key to inventing and/or understanding this algorithm is the construction of good 
“before and after” diagrams.) 


36. No; see the answer to exercise 1.2.1-15(e). 


37. If LLINK(P) = RLINK(P) = A in the representation (2), let LINK(P) = A; otherwise 
let LINK(P) = Q where NODE(Q) corresponds to NODE(LLINK(P)) and NODE(Q + 1) 
to NODE(RLINK(P)). The condition LLINK(P) or RLINK(P) = A is represented by a 
sentinel in NODE(Q) or NODE(Q+ 1) respectively. This representation uses between n 
and 2n—1 memory positions; under the stated assumptions, (2) would require 18 words 
of memory, compared to 11 in the present scheme. Insertion and deletion operations 
are approximately of equal efficiency in either representation. But this representation 
is not quite as versatile in combination with other structures. 


SECTION 2.3.2 


1. If B is empty, F(B) is an empty forest. Otherwise, F'(B) consists of a tree T plus 
the forest F(right(B)), where root(T’) = root(B) and subtrees(T) = F(left(B)). 


2. The number of zeros in the binary notation is the number of decimal points in the 
decimal notation; the exact formula for the correspondence is 


Q1.02.+++ ag 4 17019 '0...01%* 7", 


where 1° denotes a ones in a row. 


3. Sort the Dewey decimal notations for the nodes lexicographically (from left to 
right, as in a dictionary), placing a shorter sequence a1. ---.a, in front of its extensions 
@1.+++.@p.+++.@, for preorder, and behind its extensions for postorder. Thus, if we 
were sorting words instead of sequences of numbers, we would place the words cat, 
cataract in the usual dictionary order, to get preorder; we would reverse the order of 
initial subwords (cataract, cat), to get postorder. These rules are readily proved by 
induction on the size of the tree. 

4. True, by induction on the number of nodes. 

5. (a) Inorder. (b) Postorder. It is interesting to formulate rigorous induction proofs 
of the equivalence of these traversal algorithms. 

6. We have preorder(T’) = preorder(T”), and postorder(T) = inorder(T”), even if T 
has nodes with only one child. The remaining two orders are not in any simple relation; 
for example, the root of T comes at the end in one case and about in the middle in the 
other. 
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7. (a) Yes; (b) no; (c) no; (d) yes. Note that reverse preorder of a forest equals 
postorder of the left-right reversed forest (in the sense of mirror reflection). 


8. T< T’ means that either info(root(T)) < info(root(T”)), or these info’s are equal 
and the following condition holds: Suppose the subtrees of root(T) are T1,...,Tn and 
the subtrees of root(T’) are Tj,...,7%,, and let k > 0 be as large as possible such that 
T; is equivalent to T; for 1 < j < k. Then either k = n or k < n and Tpi < Thy. 


9. The number of nonterminal nodes is one less than the number of right links that 
are A, in a nonempty forest, because the null right links correspond to the rightmost 
child of each nonterminal node, and also to the root of the rightmost tree in the forest. 
(This fact gives another proof of exercise 2.3.1-14, since the number of null left links 
is obviously equal to the number of terminal nodes.) 


10. The forests are similar if and only ifn = n’ and d(u;) = d(u}), for 1 < j < n; they 
are equivalent if and only if in addition info(u;) = info(uj), 1 < j < n. The proof is 
similar to the previous proof, by generalizing Lemma 2.3.1P; let f(u) = d(u) — 1. 


11. 


(e) neg neg 


12. If INFO(Q1) # 0: Set R + COPY(P1); then if TYPE(P2) = 0 and INFO(P2) + 2, 
set R + TREE(“t”,R, TREE(INFO(P2) —1)); if TYPE(P2) # 0, set R + TREE(“t”,R, 
TREE(“—”,COPY (P2) , TREE(1))); then set Q1 + MULT (Q1 ,MULT (COPY (P2) ,R)). 

If INFO(Q) # 0: Set Q + TREE(“x”,MULT(TREE(“In”,COPY(P1)) ,Q) ,TREE(“t”, 
COPY (P1) ,COPY(P2))). 

Finally go to DIFF [4]. 


13. The following program implements Algorithm 2.3.1C with rll = P, rl2 = Q, 
rl3 = R, and with appropriate changes to the initialization and termination conditions: 


064 ST3 6F(0:2) Save contents of rI3, rI2. 

065 ST2 7F(0:2) C1. Initialize. 

066 ENT2 8F Start by creating NODE(U) with 
067 JMP 1F RLINK(U) = A. 

068 8H CON O Zero constant for initialization 
069 4H LD1 0,1(LLINK) Set P + LLINK(P) = Px. 

070 1H LD3 AVAIL R < AVAIL. 

071 J3Z OVERFLOW 

072 LDA 0,3(LLINK) 

073 STA AVAIL 

074 ST3 0,2(LLINK) LLINK (Q) < R. 

O75 ENNA 0,2 


076 STA 0,3(RLINKT) RLINK(R) + Q, RTAG(R) + 1. 


2.3.2 


O77 INCA 
O78 ENT2 
079 JAZ 
080 C2 LDA 
081 JAN 
082 LD3 
083 J3Z 
084 LDA 
085 STA 
086 LDA 
087 STA 
088 ST3 
089 C3 LDA 
090 STA 
091 LDA 
092 STA 
093 C4 LDA 
094 JANZ 
095 STZ 
096 C5 LD2N 
097 LD1 
098 J2P 
099 ENN2 
100 C6 J2NZ 
101 LD1 
102 6H ENT3 
103 7H ENT2 
14. Let 


8B 
0,3 

c3 

0,1 

c3 

AVAIL 
OVERFLOW 
0,3(LLINK) 
AVAIL 
0,2(RLINKT) 
0,3(RLINKT) 
0,2(RLINKT) 
1,1 

1,2 

0, 1 (TYPE) 
0,2(TYPE) 
0,1(LLINK) 
4B 
0,2(LLINK) 
0,2(RLINKT) 
0,1(RLINK) 
c5 

0,2 

c2 

8B (LLINK) 

* 

* 
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rA + LOC(init node) — Q. 
Set Q + R = Q*. 

To C3, the first time. 
C2. Anything to right? 
Jump if RTAG(P) = 1. 

R < AVAIL. 


Set RLINKT(R) <- RLINKT(Q). 
RLINK(Q) + R, RTAG(Q) + 0. 
C3. Copy INFO 

INFO field copied. 


TYPE field copied. 

C4. Anything to left? 

Jump if LLINK(P) Æ A. 
LLINK(Q) + A. 

C5. Advance. Q <- —RLINKT(Q). 
P + RLINK (P). 

Jump if RTAG(Q) was 1. 

Q+ —Q. 

C6. Test if complete. 


rll + location of first node created. 


Restore index registers. 
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a be the number of nonterminal (operator) nodes copied. The number of 


executions of the various lines in the previous program is as follows: 064-067, 1; 069, a; 
070-079, a + 1; 080-081, n 
n + 1; 099-100, n — a; 101-103, 1. The total time is (36n + 22)u; we use about 20% of 
the time to get available nodes, 40% to traverse, and 40% to copy the INFO and LINK 


information. 


1; 082-088, n — 1 


15. Comments are left to the reader. 


218 DIV LDA 
219 JAZ 
220 JMP 
221 ENTA 
222 ENTX 
223 JMP 
224 ENT6 
295 1H LDA 
226 JAZ 
227 JMP 
228 ST1 
229 ENTA 
230 JMP 


1,6 
1F 
COPYP2 
SLASH 
0,6 
TREE2 
0,1 

1,5 
SUB 
COPYP2 
1F(0:2) 
CON2 
TREEO 


231 
232 
233 
234 
235 
236 
237 
238 
239 
240 
241 
242 
243 
244 


1H 


1H 


ENTA 
ENTX 
JMP 
ST1 
JMP 
ENTA 
ENT1 
JMP 
ENTX 
ENT1 
ENTA 
JMP 
ENTS 
JMP 


a; 089-094, n; 095, n — a; 096-098, 


UPARROW 
* 

TREE2 
1F (0:2) 
COPYP1 
0,1 

0,5 
MULT 
0,1 
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16. Comments are left to the reader. 


245 PWR LDA 1,6 263 JMP TREEO 281 ENTA LOG 
246 JAZ 4F 264 1H ENTX * 282 JMP TREE1 
247 JMP COPYP1 265 ENTA MINUS 283 ENTA 0,1 
248 ST1 R(0:2) 266 JMP TREE2 284 ENT1 0,5 
249 LDA 0,3(TYPE) 267 5H LDX R(0:2) 285 JMP MULT 
250 JANZ 2F 268 ENTA UPARROW 286 ST1 1F(0:2) 
251 LDA 1,3 269 JMP TREE2 287 JMP COPYP1 
252 DECA 2 270 ST1 R(0:2) 288 ST1 2F(0:2) 
253 JAZ 3F 271 3H JMP COPYP2 289 JMP COPYP2 
254 INCA 1 272 ENTA 0,1 290 2H ENTX * 

255 STA CONO+1 273 R ENT1 * 291 ENTA UPARROW 
256 ENTA CONO 274 JMP MULT 292 JMP TREE2 
257 JMP TREEO 275 ENTA 0,6 298 1H ENTX * 

258 STZ CONO+1 276 JMP MULT 294 ENTA TIMES 
259 JMP 5F 277 ENT6 0,1 295 JMP TREE2 
260 2H JMP COPYP2 278 4H LDA 1,5 296 ENTS 0,1 

261 ST1 1F(0:2) 279 JAZ ADD 297 JMP ADD J 
262 ENTA CON1 280 JMP COPYP1 


17. References to early work on such problems can be found in a survey article by 
J. Sammet, CACM 9 (1966), 555-569. 


18. First set LLINK[j] < RLINK[j] <+ j for all j, so that each node is in a circular 
list of length 1. Then for j = n, n — 1, ..., 1 (in this order), if PARENT[j] = 0 set 
r < j, otherwise insert the circular list starting with j into the circular list starting with 
PARENT[7] as follows: k < PARENT[7], l + RLINK[A], i + LLINK[j], LLINK[j] + k, 
RLINK[k] + j, LLINK[/] + i, RLINK[2] + l. This works because (a) each nonroot 
node is always preceded by its parent or by a descendant of its parent; (b) nodes of 
each family appear in their parent’s list, in order of location; (c) preorder is the unique 
order satisfying (a) and (b). 


20. If u is an ancestor of v, it is immediate by induction that u precedes v in preorder 
and follows v in postorder. Conversely, suppose u precedes v in preorder and follows 
v in postorder; we must show that u is an ancestor of v. This is clear if u is the root 
of the first tree. If u is another node of the first tree, v must be also, since u follows v 
in postorder; so induction applies. Similarly if u is not in the first tree, v must not be 
either, since u precedes v in preorder. (This exercise also follows easily from the result 
of exercise 3. It gives us a quick test for ancestorhood, if we know each node’s position 
in preorder and postorder.) 


21. If NODE(P) is a binary operator, pointers to its two operands are P1 = LLINK(P) 
and P2 = RLINK(P1) = $P. Algorithm D makes use of the fact that P2s = P, so 
that RLINK(P1) may be changed to Q1, a pointer to the derivative of NODE(P1); then 
RLINK(P1) is reset later in step D3. For ternary operations, we would have, say, 
P1 = LLINK(P), P2 = RLINK(P1), P3 = RLINK(P2) = $P, so it is difficult to generalize 
the binary trick. After computing the derivative Q1, we could set RLINK (P1) + Q1 tem- 
porarily, and then after computing the next derivative Q2 we could set RLINK (Q2) + Q1 
and RLINK(P2) + Q2 and reset RLINK (P1) + P2. But this is certainly inelegant, and it 
becomes progressively more so as the degree of the operator becomes higher. Therefore 
the device of temporarily changing RLINK(P1) in Algorithm D is definitely a trick, 
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not a technique. A more aesthetic way to control a differentiation process, because it 
generalizes to operators of higher degree and does not rely on isolated tricks, can be 
based on Algorithm 2.3.3F; see exercise 2.3.3-3. 


22. From the definition it follows immediately that the relation is transitive; that is, 
if TCT’ and T’ CT” then T C T”. (In fact the relation is easily seen to be a partial 
ordering.) If we let f be the function taking nodes into themselves, clearly I(T) C T 
and r(T) C T. Therefore if T C I(T’) or T C r(T’) we must have T C T'. 

Suppose fı and f, are functions that respectively show I(T) C I(T’) and r(T) C 
r(T'). Let f(u) = fi(u) if u is in I(T), f(u) = root(T’) if u is root(T), otherwise 
f(u) = fr(u). Now it follows easily that f shows T C T’; for example, if we let r’(T) 
denote r(T) \ root(T) we have preorder(T) = root(T) preorder(I(T)) preorder(r’(T)); 
preorder(7”) = f(root(T)) preorder(I(T’)) preorder(r’(T")). 

The converse does not hold: Consider the subtrees with roots b and b’ in Fig. 25. 


SECTION 2.3.3 


1. Yes, we can reconstruct them just as (3) is deduced from (4), but interchanging 
LTAG and RTAG, LLINK and RLINK, and using a queue instead of a stack. 


2. Make the following changes in Algorithm F: Step F1, change to “last node of the 
forest in preorder.” Step F2, change “f(aa),...,f(a1)” to “f(ax1),..., f(wa)” in two 
places. Step F4, “If P is the first node in preorder, terminate the algorithm. (Then the 
stack contains f(root(T1)),..., f(root(Tin)), from top to bottom, where T1,...,Tm are 
the trees of the given forest, from left to right.) Otherwise set P to its predecessor in 
preorder (P + P — c in the given representation), and return to F2.” 


3. In step D1, also set S + A. (S is a link variable that links to the top of the 
stack.) Step D2 becomes, for example, “If NODE(P) denotes a unary operator, set 
Q + S, S «+ RLINK(Q), P1 «+ LLINK(P); if it denotes a binary operator, set Q + S, 
Qi «+ RLINK(Q), S ¢ RLINK(Q1), P1 < LLINK(P), P2 « RLINK(P1). Then perform 
DIFF[TYPE(P)].” Step D3 becomes “Set RLINK(Q) «+ S, S 4+ Q.” Step D4 becomes 
“Set P 4+ Ps.” The operation LLINK(DY) + Q may be avoided in step D5 if we assume 
that S = LLINK(DY). This technique clearly generalizes to ternary and higher-order 
operators. 


4. A representation like (10) takes n — m LLINKs and n + (n — m) RLINKs. The 
difference in total number of links is n — 2m between the two forms of representation. 
Arrangement (10) is superior when the LLINK and INFO fields require about the same 
amount of space in a node and when m is rather large, namely when the nonterminal 
nodes have rather large degrees. 


5. It would certainly be silly to include threaded RLINKs, since an RLINK thread just 
points to PARENT anyway. Threaded LLINKs as in 2.3.2-(4) would be useful if it is 
necessary to move leftward in the tree, for example if we wanted to traverse a tree in 
reverse postorder, or in family order; but these operations are not significantly harder 
without threaded LLINKs unless the nodes tend to have very high degrees. 


6. L1. Set P + FIRST, FIRST + A. 
L2. If P = A, terminate. Otherwise set Q + RLINK(P). 


L3. If PARENT(P) = A, set RLINK(P) + FIRST, FIRST + P; otherwise set R < 
PARENT(P), RLINK(P) «+ LCHILD(R), LCHILD(R) + P. 


L4. Set P+ Q and return to L2. J 
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7. {1,5}{2, 3,4, 7}{6, 8, 9}. 
8. Perform step E3 of Algorithm E, then test if 7 = k. 


9. PARENT[A]: 5 0 2 2 0 8 2 2 8 
k: 1 2 3 4 5 67 8 9 


(2) © 
J O OR © 
© © 


10. One idea is to set PARENT of each root node to the negative of the number of 
nodes in its tree (these values being easily kept up to date); then if |PARENT[j]| > 
|PARENT [k]| in step E4, the roles of j and k are interchanged. This technique (due to 
M. D. McIlroy) ensures that each operation takes O(log n) steps. 

For still more speed, we can use the following suggestion due to Alan Tritter: In 
step E4, set PARENT[2z] + k for all values x 4 k that were encountered in step E3. 
This makes an extra pass up the trees, but it collapses them so that future searches 
are faster. (See Section 7.4.1.) 


11. It suffices to define the transformation that is done for each input (P, j, Q, k): 


T1. If PARENT(P) Æ A, set j + j +DELTA(P), P 4 PARENT(P), and repeat this 
step. 

T2. If PARENT(Q) # A, set k + k + DELTA(Q), Q + PARENT(Q), and repeat this 
step. 


T3. If P = Q, check that 7 = k (otherwise the input erroneously contains con- 
tradictory equivalences). If P Æ Q, set DELTA(Q) + j — k, PARENT(Q) < P, 
LBD(P) «+ min(LBD(P), LBD(Q) + DELTA(Q)), and UBD(P) + max(UBD(P), 
UBD(Q) + DELTA(Q)). I 


Note: It is possible to allow the ARRAY X[l:u] declarations to occur intermixed with 
equivalences, or to allow assignment of certain addresses of variables before others are 
equivalenced to them, etc., under suitable conditions that are not difficult to under- 
stand. For further development of this algorithm, see CACM 7 (1964), 301-303, 506. 


12. (a) Yes. (If this condition is not required, it would be possible to avoid the loops 
on S that appear in steps A2 and A9.) (b) Yes. 


13. The crucial fact is that the UP chain leading upward from P always mentions the 
same variables and the same exponents for these variables as the UP chain leading 
upward from Q, except that the latter chain may include additional steps for variables 
with exponent zero. (This condition holds throughout most of the algorithm, except 
during the execution of steps A9 and A10.) Now we get to step A8 either from A3 or 
from A10, and in each case it was verified that EXP(Q) # 0. Therefore EXP(P) Æ 0, 
and in particular it follows that P 4 A, Q Æ A, UP(P) Æ A, UP(Q) Æ A; the result 
stated in the exercise now follows. Thus the proof depends on showing that the UP 
chain condition stated above is preserved by the actions of the algorithm. 


14, 15. See Martin Ward and Hussain Zedan, “Provably correct derivation of algo- 
rithms using FermaT,” Formal Aspects of Computing 27 (2015), to appear. 
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16. We prove (by induction on the number of nodes in a single tree T) that if P is 
a pointer to T, and if the stack is initially empty, steps F2 through F4 will end with 
the single value f(root(T)) on the stack. This is true for n = 1. If n > 1, there are 
0 < d = DEGREE(root(T)) subtrees T}, ...,Ta; by induction and the nature of a stack, 
and since postorder consists of T), ..., T4 followed by root(T), the algorithm computes 
f(Tı),--., f(Ta), and then f(root(T)), as desired. The validity of Algorithm F for 
forests follows. 

17. G1. Set the stack empty, and let P point to the root of the tree (the last node in 

postorder). Evaluate f(NODE(P)). 


G2. Push DEGREE (P) copies of f(NODE(P)) onto the stack. 


G3. If P is the first node in postorder, terminate the algorithm. Otherwise set P 
to its predecessor in postorder (this would be simply P +P — c in (9)). 
G4. Evaluate f(NODE(P)) using the value at the top of the stack, which is equal 
to f(NODE(PARENT(P))). Pop this value off the stack, and return to G2. J 
Note: An algorithm analogous to this one can be based on preorder instead of postorder 
as in exercise 2. In fact, family order or level order could be used; in the latter case we 
would use a queue instead of a stack. 
18. The INFO1 and RLINK tables, together with the suggestion for computing LTAG in 
the text, give us the equivalent of a binary tree represented in the usual manner. The 
idea is to traverse this tree in postorder, counting degrees as we go: 
P1. Let R, D, and I be stacks that are initially empty; then set R= n +1, D < 0, 
j +0, k <0. 
P2. If top(R) > j + 1, go to P5. (If an LTAG field were present, we could have 
tested LTAG[7] = 0 instead of top(R) > j + 1.) 
P3. If I is empty, terminate the algorithm; otherwise set i < I, k + k +1, 
INF02[k] «+ INF01[i], DEGREE[k] < D. 
P4. If RLINK[i] = 0, go to P3; otherwise delete the top of R (which will equal 
RLINK[i]). 
P5. Set top(D) + top(D) + 1, j + j +1, I < j, D <0, and if RLINK[j] # 0 set 
R < RLINK[j]. Go to P2. I 
19. (a) This property is equivalent to saying that SCOPE links do not cross each other. 
(b) The first tree of the forest contains dı+1 elements, and we can proceed by induction. 
(c) The condition of (a) is preserved when we take minima. 

Notes: By exercise 2.3.2—20, it follows that dıdə2...dn can also be interpreted in 
terms of inversions: If the kth node in postorder is the pth node in preorder, then dx 
is the number of elements > k that appear to the left of k in pıp2... Pn. 

A similar scheme, in which we list the number of descendants of each node in 
postorder of the forest, leads to sequences of numbers c1c2...Cn characterized by the 
properties (i) 0 < cx < k and (ii) k > j > k — cp implies j — cj > k — cp. Algorithms 
based on such sequences have been investigated by J. M. Pallo, Comp. J. 29 (1986), 
171-175. Notice that cw is the size of the left subtree of the kth node in symmetric 
order of the corresponding binary tree. We can also interpret dx as the size of the right 
subtree of the Ath node in symmetric order of a suitable binary tree, namely the binary 
tree that corresponds to the given forest by the dual method of exercise 2.3.2—5. 

The relation dẹ < dh for 1 < k < n defines an interesting lattice ordering of forests 
and binary trees, first introduced in another way by D. Tamari [Thèse (Paris, 1951)]; 
see exercise 6.2.3-32. 
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SECTION 2.3.4.1 

1. (B, A, C, D, B), (B, A, C, D, E, B), (B, D, C, A, B), (B, D, E, B), (B, E, D, B), 
(B, E, D, C, A, B): 

2. Let (Vo, Vi,..., Vn) be a walk of smallest possible length from V to V’. If now 
V; = Vp for some j < k, then (Vo, ..., Vj, Vk+1,---, Vn) would be a shorter walk. 


3. (The fundamental path traverses e3 and e4 once, but cycle C2 traverses them —1 
times, giving a net total of zero.) Traverse the following edges: e1, €2, e6, €7, €9, €10, 
€11, €12, €14. 

4. If not, let G” be the subgraph of G’ obtained by deleting each edge e; for which 
Ej = 0. Then G” is a finite graph that has no cycles and at least one edge, so by 
the proof of Theorem A there is at least one vertex, V, that is adjacent to exactly one 
other vertex, V’. Let ej be the edge joining V to V’; then Kirchhoff’s equation (1) at 
vertex V is FE; = 0, contradicting the definition of G”. 

5. A=1+ Es, B= 1+ Es Bo, C = 1 + Es, D = 1 + Es — Es, E = 1 + Fiz — Fai, 
F=14 EY’; + Biz Fa, G = 14+ Eih, H = E7 Fai, J= Fiz, k= Eih + Foo, 
L = E\74+ Ejo + E2 — En, P = E17 + Ez — En, Q = Eno, R = Ey7 — En, S = Eos. 
Note: In this case it is also possible to solve for E2, Fs5,..., E25 in terms of A, B,..., 5; 
hence there are nine independent solutions, explaining why we eliminated six variables 
in Eq. 1.3.3-(8). 

6. (The following solution is based on the idea that we may print out each edge that 
does not make a cycle with the preceding edges.) Use Algorithm 2.3.3E, with each pair 
(ai, bi) representing a; = b; in the notation of that algorithm. The only change is to 
print (ai, bi) if j Æ k in step E4. 

To show that this algorithm is valid, we must prove that (a) the algorithm prints 
out no edges that form a cycle, and (b) if G contains at least one free subtree, the 
algorithm prints out n — 1 edges. Define j = k if there exists a path from V; to Vz 
or if j = k. This is clearly an equivalence relation, and moreover j = k if and only 
if this relation can be deduced from the equivalences a1 = bi, ..., Gm = bm. Now 
(a) holds because the algorithm prints out no edges that form a cycle with previously 
printed edges; (b) is true because PARENT[k] = 0 for precisely one k if all vertices are 
equivalent. 

A more efficient algorithm can, however, be based on depth-first search; see 
Algorithm 2.3.5A and Section 7.4.1. 


7. Fundamental cycles: Co = eo + e1 + e4 + e9 (fundamental path is e1 + e4 + e9); 
Cs = e5 + e3 + e2; Ce = es — €2 + e4; C7 = e7 — e4 — €3; Cg = eg — eg — es — e3. 
Therefore we find Ey = 1, Ez = Es Ee, E3 = Es Er Es, E4 = 1+ Es = Er = Es, 
Ey = 1 — Es. 

8. Each step in the reduction process combines two arrows e; and ej that start at 
the same box, and it suffices to prove that such steps can be reversed. Thus we are 
given the value of e; +e; after combination, and we must assign consistent values to e; 
and ej before the combination. There are three essentially different situations: 


Case 1 Case 2 Case 3 
Before 


B 
B! 
a ej al a” a a! 
ay 8 Ea y BK 
eA Ez |B] B' Sj B” B ej |B| B' 
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After 
a ey +e; 
a ei+ej al a EN ail 
ae e AD 
B 2 p’ B p 


Here A, B, and C stand for vertices or supervertices, and the a’s and ’s stand for the 
other given flows besides e; + e;; these flows may each be distributed among several 
edges, although only one is shown. In Case 1 (e; and e; lead to the same box), we may 
choose e; arbitrarily, then ej + (e; + ej) — e:i. In Case 2 (e; and ej lead to different 
boxes), we must set e; + 8’ — a’, ej + p” — a”. In Case 3 (e; is a loop but ej is 
not), we must set ej + 8’ — a’, ei + (ei +e;) — ej. In each case we have reversed the 
combination step as desired. 

The result of this exercise essentially proves that the number of fundamental cycles 
in the reduced flow chart is the minimum number of vertex flows that must be measured 
to determine all the others. In the given example, the reduced flow chart reveals that 
only three vertex flows (e.g., a, c, d) need to be measured, while the original chart 
of exercise 7 has four independent edge flows. We save one measurement every time 
Case 1 occurs during the reduction. 

A similar reduction procedure could be based on combining the arrows flowing into 
a given box, instead of those flowing out. It can be shown that this would yield the 
same reduced flow chart, except that the supervertices would contain different names. 

The construction in this exercise is based on ideas due to Armen Nahapetian and 
F. Stevenson. For further comments, see A. Nahapetian, Acta Informatica 3 (1973), 
37-41; D. E. Knuth and F. Stevenson, BIT 13 (1973), 313-322. 


9. Each edge from a vertex to itself becomes a “fundamental cycle” all by itself. If 
there are k + 1 edges e,e’,.. . et") between vertices V and V’, make k fundamental 
cycles e' +e, ..., eP +e (choosing + or — according as the edges go in the opposite 
or the same direction), and then proceed as if only edge e were present. 

Actually this situation would be much simpler conceptually if we had defined a 
graph in such a way that multiple edges are allowed between vertices, and edges are 
allowed from a vertex to itself; paths and cycles would be defined in terms of edges 
instead of vertices. Such a definition is, in fact, made for directed graphs in Section 
2.3.4.2. 


10. If the terminals have all been connected together, the corresponding graph must 
be connected in the technical sense. A minimum number of wires will involve no cycles, 
so we must have a free tree. By Theorem A, a free tree contains n — 1 wires, and a 
graph with n vertices and n — 1 edges is a free tree if and only if it is connected. 


11. It is sufficient to prove that when n > 1 and c(n — 1, n) is the minimum of the 
c(i, n), there exists at least one minimum cost tree in which T,,-1 is wired to Tn. (For, 
any minimum cost tree with n > 1 terminals and with T;,_1 wired to Ta must also be a 
minimum cost tree with n — 1 terminals if we regard T;,,-1 and Tn as “common,” using 
the convention stated in the algorithm.) 

To prove the statement above, suppose we have a minimum cost tree in which 
Tn-1 is not wired to Tn. If we add the wire T;,-1 — Tn we obtain a cycle, and any 
of the other wires in that cycle may be removed; removing the other wire touching Th 
gives us another tree, whose total cost is not greater than the original, and T,-1 — Tn 
appears in that tree. 
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12. Keep two auxiliary tables, a(i) and b(i), for 1 < i < n, representing the fact that 
the cheapest connection from T; to a chosen terminal is to Teq), and its cost is a(i); 
initially a(i) = c(i, n) and b(i) = n. Then do the following operation n — 1 times: Find 
i such that a(i) = mini<;<n a(j); connect T; to Ty); for 1 < j < n if c(t, j) < a(j) set 
a(j) < c(i, j) and b(j) < i; and set a(i) + oo. Here c(i, j) means c(j,i) when j < i. 
(It is somewhat more efficient to avoid the use of oo, keeping instead a one- 
way linked list of those j that have not yet been chosen. With or without this 
straightforward improvement, the algorithm takes O(n”) operations.) See also E. W. 
Dijkstra, Proc. Nederl. Akad. Wetensch. A63 (1960), 196-199; D. E. Knuth, The 
Stanford GraphBase (New York: ACM Press, 1994), 460-497. Significantly better 
algorithms to find a minimum-cost spanning tree are discussed in Section 7.5.4. 


13. If there is no path from V; to V;, for some i # j, then no product of the 
transpositions will move i to j. So if all permutations are generated, the graph must be 
connected. Conversely if it is connected, remove edges if necessary until we have a free 
tree. Then renumber the vertices so that V» is adjacent to only one other vertex, namely 
Vn—1. (See the proof of Theorem A.) Now the transpositions other than (n—1 n) form 
a free tree with n — 1 vertices; so by induction if 7 is any permutation of {1,2,...,n} 
that leaves n fixed, 7 can be written as a product of those transpositions. If 7 moves 
n to j then 7(j n—1)(n—1 n) = p fixes n; hence 7 = p(n—1 n)(j n—1) can be written 
as a product of the given transpositions. 


SECTION 2.3.4.2 


1. Let (e1,...,@n) be an oriented walk of smallest possible length from V to V’. If 
now init(e;) = init(e,) for j < k, (e1,...,€j;~1,k,---,€n) would be a shorter walk; a 
similar argument applies if fin(e;) = fin (ex) for j < k. Hence (e1,..., én) is simple. 

2. Those cycles in which all signs are the same: Co, Cs, Cis, C17, Cio, C20. 

3. For example, use three vertices A, B, C, with arcs from A to B and A to C. 


4. If there are no oriented cycles, Algorithm 2.2.3T topologically sorts G. If there 
is an oriented cycle, topological sorting is clearly impossible. (Depending on how this 
exercise is interpreted, oriented cycles of length 1 could be excluded from consideration.) 

5. Let k be the smallest integer such that fin(e,) = init(e;) for some j < k. Then 
(e;,.--,e%) is an oriented cycle. 

6. False (on a technicality), just because there may be several different arcs from one 
vertex to another. 

7. True for finite directed graphs: If we start at any vertex V and follow the only 
possible oriented path, we never encounter any vertex twice, so we must eventually 
reach the vertex R (the only vertex with no successor). For infinite directed graphs the 
result is obviously false since we might have vertices R, Vi, V2, V3,... and arcs from Vj 
to Vj41 for j > 1. 

9. Allarcs point upward. G 
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10. G1. Set k + Ply], Ply] + 0. 
G2. If k = 0, stop; otherwise set m + P[k], P[k] + j, j + k, k + m, and repeat 
step G2. J 
11. This algorithm combines Algorithm 2.3.3E with the method of the preceding 
exercise, so that all oriented trees have arcs that correspond to actual arcs in the 
directed graph; S[j] is an auxiliary table that tells whether an arc goes from j to P[j] 
(S[j] = +1) or from P[j] to j (S[j] = —1). Initially P[1] = --- = P[n] = 0. The 
following steps may be used to process each arc (a, b): 
C1. Set j H+ a, k + Ply], P[j] + 0, s + S[y]. 
C2. If k = 0, go to C3; otherwise set m 4+ P[k], t + S[k], P[k] + j, S[k] + —s, 
s + t, j 4+ k, k 4+ m, and repeat step C2. 
C3. (Now a appears as the root of its tree.) Set j « b, and then if P[j] # 0 
repeatedly set j + P[j] until P[j] = 0. 
C4. If j = a, go to C5; otherwise set Pla] + b, S[a] + +1, print (a,b) as an arc 
belonging to the free subtree, and terminate. 
C5. Print “CYCLE” followed by “(a,b)”. 
C6. If P[b] = 0 terminate. Otherwise if S[b] = +1, print “+(b, P[b])”, else print 
“_(P[b], b)”; set b ~ P[b] and repeat step C6. J 
Note: This algorithm will take at most O(m log n) steps if we incorporate the suggestion 
of McIlroy in answer 2.3.3-10. But there is a much better solution that needs only O(m) 


steps: Use depth-first search to construct a “palm tree,” with one fundamental cycle 
for each “frond” [R. E. Tarjan, SICOMP 1 (1972), 146-150]. 


12. It equals the in-degree; the out-degree of each vertex can be only 0 or 1. 


13. Define a sequence of oriented subtrees of G as follows: Go is the vertex R alone. 
G41 is Gz, plus any vertex V of G that is not in Gk but for which there is an arc from 
V to V’ where V’ is in Gx, plus one such arc e[V] for each such vertex. It is immediate 
by induction that G; is an oriented tree for all k > 0, and that if there is an oriented 
path of length k from V to R in G then V is in Gk. Therefore Go, the set of all V 
and e[V] in any of the Gx, is the desired oriented subtree of G. 


14. (e12, €20, €00, €01; €10; €01, €12, €22, €21), (e12, €20, €00, €01, €12; €22; €21, €10; €01); 
(e12, €20, €01; €10, €00; €01; €12, €22; €21), (e12, €20, €01, €12, €22, €21, €10, €00; €01); 
(e12, €22, €20, €00; €01; €10 €01, €12; €21), (e12, €22, €20, €00, €01, €12; €21, €10; €01); 
(€12, €22, €20; e01, €10; €00; €01; €12; €21)» (e12, €22, €20, €01, €12; €21; €10, €00; €01); 

in lexicographic order; the eight possibilities come from the independent choices of 

which of eoo or €01, €10 OF €42, €20 OF e22, Should precede the other. 


15. True for finite graphs: If it is connected and balanced and has more than one 
vertex, it has an Eulerian trail that touches all the vertices. (But false in general.) 


16. Consider the directed graph G with vertices Vi, ..., Vig and with an arc from V; 
to Vk for each k in pile j; this graph is balanced. Winning the game is equivalent to 
tracing out an Eulerian trail in G, because the game ends when the fourth arc to Vi3 
is encountered (namely, when the fourth king turns up). Now if the game is won, the 
stated digraph is an oriented subtree by Lemma E. Conversely if the stated digraph is 
an oriented tree, the game is won by Theorem D. 


17. b This answer can be obtained, as the author first obtained it, by laborious 


enumeration of oriented trees of special types and the application of generating func- 
tions, etc., based on the methods of Section 2.3.4.4. But such a simple answer deserves 
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a simple, direct proof, and indeed there is one [see Tor B. Staver, Norsk Matematisk 
Tidsskrift 28 (1946), 88-89]. Define an order for turning up all cards of the deck, as 
follows: Obey the rules of the game until getting stuck, then “cheat” by turning up the 
first available card (find the first pile that is not empty, going clockwise from pile 1) 
and continue as before, until eventually all cards have been turned up. The cards in 
the order of turning up are in completely random order (since the value of a card need 
not be specified until after it is turned up). So the problem is just to calculate the 
probability that in a randomly shuffled deck the last card is a king. More generally the 
probability that k cards are still face down when the game is over is the probability that 


the last king in a random shuffle is followed by k cards, namely 4! ors K) 48l. Hence a 


521 

person playing this game without cheating will turn up an average of eae 42.4 cards 
per game. Note: Similarly, it can be shown that the probability that the player will 
have to “cheat” k times in the process described above is exactly given by the Stirling 
number [a 7/131 (See Eq. 1.2.10-(9) and exercise 1.2.10-7; the case of a more general 


card deck is considered in exercise 1.2.10-18.) 


18. (a) If there is a cycle (Vo, Vi,..., Ve), where necessarily 3 < k < n, the sum of the 
k rows of A corresponding to the k edges of this cycle, with appropriate signs, is a row 
of zeros; so if G is not a free tree the determinant of Ao is zero. 

But if G is a free tree we may regard it as an ordered tree with root Vo, and we 
can rearrange the rows and columns of Ao so that columns are in preorder and so that 
the kth row corresponds to the edge from the kth vertex (column) to its parent. Then 
the matrix is triangular with +1’s on the diagonal, so the determinant is +1. 

(b) By the Binet-Cauchy formula (exercise 1.2.3-46) we have 


det Aj Ao = 5 (det Ai,...in)” 


1i < <in Im 


where A; ...i„ represents a matrix consisting of rows i1, .. . , în of Ao (thus corresponding 
to a choice of n edges of G). The result now follows from (a). 
[See S. Okada and R. Onodera, Bull. Yamagata Univ. 2 (1952), 89-117.] 


19. (a) The conditions aoo = 0 and aj; = 1 are just conditions (a), (b) of the definition 
of oriented tree. If G is not an oriented tree there is an oriented cycle (by exercise 7), 
and the rows of Ao corresponding to the vertices in this oriented cycle will sum to a 
row of zeros; hence det Ao = 0. If G is an oriented tree, assign an arbitrary order to the 
children of each family and regard G as an ordered tree. Now permute rows and columns 
of Ao until they correspond to preorder of the vertices. Since the same permutation 
has been applied to the rows as to the columns, the determinant is unchanged; and the 
resulting matrix is triangular with +1 in every diagonal position. 

(b) We may assume that ao; = 0 for all j, since no arc emanating from Vo can 
participate in an oriented subtree. We may also assume that ajj > 0 for all j > 1 since 
otherwise the whole jth row is zero and there obviously are no oriented subtrees. Now 
use induction on the number of arcs: If ajj > 1 let e be some arc leading from Vj; let 
Bo be a matrix like Ag but with arc e deleted, and let Co be the matrix like Ag but 
with all arcs except e that lead from V; deleted. Example: If Ao = (3 Ea j = 1, 
and e is an arc from Vi to Vo, then Bo = ie =) Co = (a ae In general we have 
det Ap = det Bo + det Co, since the matrices agree in all rows except row j, and Ag is 
the sum of Bo and Co in that row. Moreover, the number of oriented subtrees of G 
is the number of subtrees that do not use e (namely, det Bo, by induction) plus the 
number that do use e (namely, det Co). 
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Notes: The matrix A is often called the Laplacian of the graph, by analogy with a 
similar concept in the theory of partial differential equations. If we delete any set S of 
rows from the matrix A, and the same set of columns, the determinant of the resulting 
matrix is the number of oriented forests whose roots are the vertices {V; | k € S} and 
whose arcs belong to the given digraph. The matrix tree theorem for oriented trees 
was stated without proof by J. J. Sylvester in 1857 (see exercise 28), then forgotten for 
many years until it was independently rediscovered by W. T. Tutte [Proc. Cambridge 
Phil. Soc. 44 (1948), 463-482, §3]. The first published proof in the special case of 
undirected graphs, when the matrix A is symmetric, was given by C. W. Borchardt 
[Crelle 57 (1860), 111-121]. Several authors have ascribed the theorem to Kirchhoff, 
but Kirchhoff proved a quite different (though related) result. 


20. Using exercise 18 we find B = A? Ag. Or, using exercise 19, B is the matrix Ao for 
the directed graph G” with two arcs (one in each direction) in place of each edge of G; 
each free subtree of G corresponds uniquely to an oriented subtree of G’ with root Vo, 
since the directions of the arcs are determined by the choice of root. 


21. Construct the matrices A and A” as in exercise 19. For the example graphs G 


and Gram Figs- a0 and 3i; [00] [10] [20] [01] [01] [21] [12] [12] [22] 


00 2 0 0; -1 -1 0 0 0 0 
10 -1 3 Of -1 -1 0 0 0 0 
20 -1 0 3|-—-1 -1 0 0 0 0 
2-2 0 01 0 -1 0 3 0 Oj}; -1 -1 0 
Ass ft 3—2], APS fo 0-1 0| 0 3 2) 1 -1 0 
—1 —1 2 21 0- =l 0 0 0 3 | —1 —1 0 
12 0 0 -1 0 0 -1 3 0 -1 
12 0 oO -1 0 0 -1 0 3 -1 
22 0 0 -1 0 oO -1 0 0 2 
Add the indeterminate A to every diagonal element of A and A*. If t(G) and t(G*) are 


the numbers of oriented subtrees of G and G*, we then have det A = At(G) + O(A°), 
det A* = \t(G*) + O(A?). (The number of oriented subtrees of a balanced graph is the 
same for any given root, by exercise 22, but we do not need that fact.) 

If we group vertices V;x for equal k the matrix A* can be partitioned as shown 
above. Let B,x: be the submatrix of A* consisting of the rows for Vj, and the columns 
for Vy’, for all j and j’ such that Vj, and Vj, are in G*. By adding the 2nd, ..., 
mth columns of each submatrix to the first column and then subtracting the first row 


of each submatrix from the 2nd, ..., mth rows, the matrix A* is transformed so that 
Akl * «1. * A+Gkk * wee * 
0 oO... 0 P 0 Atm... 0 
Burr = $ : 2 š for k x k > Bkk = ż Š% . 
0 oO... 0 0 0 ... Atm 


The asterisks in the top rows of the transformed submatrices turn out to be irrelevant, 
because the determinant of A* is now seen to be (A+ m)'"—)” times 


A + aoo ao1 sh @0(n—1) 
aio A+ an ~. @1(n-1) 2 
det : : . : = At(G) + O(A). 


Qn—1)0 An—1)1 ++» AF anin) 
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Notice that when n = 1 and there are m arcs from Vọ to itself, we find in particular 
that exactly m’—! oriented trees are possible on m labeled nodes. This result will be 
obtained by quite different methods in Section 2.3.4.4. 

This derivation can be generalized to determine the number of oriented subtrees 
of G* when G is an arbitrary directed graph; see R. Dawson and I. J. Good, Ann. 
Math. Stat. 28 (1957), 946-956; D. E. Knuth, Journal of Combinatorial Theory 3 
(1967), 309-314. An alternative, purely combinatorial proof has been given by J. B. 
Orlin, Journal of Combinatorial Theory B25 (1978), 187—198. 


22. The total number is (o1 + --- + on) times the number of Eulerian trails starting 
with a given edge e1, where init(e1) = Vi. Each such trail determines an oriented 
subtree with root V; by Lemma E, and for each of the T oriented subtrees there are 
[]j_1(¢; — 1)! walks satisfying the three conditions of Theorem D, corresponding to 
the different order in which the arcs {e | init(e) = Vj, e # e[Vj], e # e1} are entered 
into P. (Exercise 14 provides a simple example.) 


23. Construct the directed graph Gk with mT! vertices as in the hint, and denote 
by [v1,...,2x] the arc mentioned there. For each function that has maximum period 
length, we can define a unique corresponding Eulerian trail, by letting f(x1,..., £k) = 
zk+1 if arc [x1,..., £k] is followed by [x2,...,v%+41]. (We regard Eulerian trails as 
being the same if one is just a cyclic permutation of the other.) Now G, = G_, in 

k-1 k— 


—m 


. 2 . . 
the sense of exercise 21, so Gy has m”™ times as many oriented subtrees as 


K . k-1_ . k-1_ M . 
Gx-1; by induction Gk has m™ l oriented subtrees, and m”™ k with a given 


root. Therefore by exercise 22 the number of functions with maximum period, namely 
the number of Eulerian trails of Gy, starting with a given arc, is mE (mh. [For 
m = 2 this result is due to C. Flye Sainte-Marie, L’Intermédiaire des Mathématiciens 
1 (1894), 107-110.] 


24. Define a new directed graph having E; copies of ej, for 0 < j < m. This graph 
is balanced, hence it contains an Eulerian trail (e0,...) by Theorem G. The desired 
oriented walk comes by deleting the edge eo from this Eulerian trail. 


25. Assign an arbitrary order to all arcs in the sets J; = {e | init(e) = Vj} and 
Fj = {e | fin(e) = Vj}. For each arc e in Ij, let ATAG(e) = 0 and ALINK(e) = e’ if 
e’ follows e in the ordering of Ij; also let ATAG(e) = 1 and ALINK(e) = e’ if e is last 
in Ij and e’ is first in F;. Let ALINK(e) = A in the latter case if Fj is empty. Define 
BLINK and BTAG by the same rules, reversing the roles of init and fin. 


Examples (using alphabetic order in each set of arcs): 


M wy MS 
=z oe Ss 
Som MS oy HERAF 
o Aa žl 4 & z 3% mMM 
soak 
adatom «a a c 0 b 1 
a d 1 b 0 b a 1 d 0 
b Alec 0 c e 0 a 1 
c f 1 ri dh 0 f 0 
d Ale O e g0 f 1 
e A 1 a 1 f j 0 d 1 
f Alec 1 g c 1 h 0 
ra 1l A 1 h b 1 j 0 
j e 1 A 1 
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Note: If in the oriented tree representation we add another arc from H to itself, 
we get an interesting situation: Either we get the standard conventions 2.3.1—(8) with 
LLINK, LTAG, RLINK, RTAG interchanged in the list head, or (if the new arc is placed 
last in the ordering) we get the standard conventions except RTAG = 0 in the node 
associated with the root of the tree. 

This exercise is based on an idea communicated to the author by W. C. Lynch. 
Can tree traversal algorithms like Algorithm 2.3.15 be generalized to classes of digraphs 
that are not oriented trees, using such a representation? 


27. Let aij be the sum of p(e) over all arcs e from V; to Vj. We are to prove that 
t; = J pizti for all j. Since $7, aj; = 1, we must prove that 37, ajit; = 0, aajti. 
But this is not difficult, because both sides of the equation represent the sum of all 
products p(e1)...p(én) taken over subgraphs {e1,..., en} of G such that init(e;) = V; 
and such that there is a unique oriented cycle contained in {e1,..., €n }, where this cycle 
includes V;. Removing any arc of the cycle yields an oriented tree; the left-hand side of 
the equation is obtained by factoring out the arcs that leave V;, while the right-hand 
side corresponds to those that enter Vj. 
In a sense, this exercise is a combination of exercises 19 and 26. 


28. Every term in the expansion is Qip; .--@mpm01q, ---Ongn, Where 0 < p; < n for 
1<i<mand0<q<™m for 1 <j <n, times some integer coefficient. Represent 
this product as a directed graph on the vertices {0,ui,...,Um,U1,---,;Un}, with arcs 
from u; to vp, and from vj to ug, where uo = vo = 0. 

If the digraph contains a cycle, the integer coefficient is zero. For each cycle 
corresponds to a factor of the form 


Qiojo Djoir Ginga +++ Cin—rin—1 Ojn—ri0 (*) 
where the indices (to, 71,..., 7-1) are distinct and so are the indices (jo, j1,..-,jx—1)- 
The sum of all terms containing (*) as a factor is (*) times the determinant obtained 
by setting ay; 4 [J=] for 0 < j < nand by; © [i=in moak] for 0 <i m, 
for 0 < l < k, leaving the variables in the other m + n — 2k rows unchanged. This 
determinant is identically zero, because the sum of rows to, i1, ..., ik—1 in the top 
section equals the sum of rows jo, jı, ---,; Je—1 in the bottom section. 

On the other hand, if the directed graph contains no cycles, the integer coefficient 
is +1. This follows because each factor aip; and bjg; must have come from the diagonal 
of the determinant: If any off-diagonal element aigjọ is chosen in row to of the top 
section, we must choose some off-diagonal b;,;, from row jo of the bottom section, 
hence we must choose some off-diagonal a;,;, from row 71 of the top section, etc., 
forcing a cycle. 

Thus the coefficient is +1 if and only if the corresponding digraph is an oriented 
tree with root 0. The number of such terms (hence the number of such oriented trees) 
is obtained by setting each a;j and bj; to 1; for example, 


4 O0 1 1 1 4 0 1 1l 1 4 0 3 1 1 
0 4 1 1 1 —4 4 0 0 0 04000 
det | 1 1 3 0 O | =det 1 1 3 0 0]=det]2 1 3 0 0 
1 103 0 0 0 -3 3 0 00 0 3 0 
1 1 0 0 3 0 0 -3 0 3 00003 
4 3 
= aet (3 g) ded. 


In general we obtain det("** a F H=. (m HY 
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Notes: J. J. Sylvester considered the special case m = n and aio = a20 = 

- = amo = 0 in Quarterly J. of Pure and Applied Math. 1 (1857), 42-56, where 

he conjectured (correctly) that the total number of terms is then n”(n + 1)"~'. He 

also stated without proof that the (n+ 1)"~' nonzero terms present when aij = 6:; 

correspond to all connected cycle-free graphs on {0,1,...,7}. In that special case, he 
reduced the determinant to the form in the matrix tree theorem of exercise 19, e.g., 


bio + b12 + b13 —bi2 —bi3 
det — b21 b20 + b21 + b23 —b23 
—b31 —b32 b30 + b31 + b32 


Cayley quoted this result in Crelle 52 (1856), 279, ascribing it to Sylvester; thus it is 
ironic that the theorem about the number of such graphs is often attributed to Cayley. 

By negating the first m rows of the given determinant, then negating the first m 
columns, we can reduce this exercise to the matrix tree theorem. 

[Matrices having the general form considered in this exercise are important in 
iterative methods for the solution of partial differential equations, and they are said 
to have “Property A.” See, for example, Louis A. Hageman and David M. Young, 
Applied Iterative Methods (Academic Press, 1981), Chapter 9.] 


SECTION 2.3.4.3 
1. The root is the empty sequence; arcs go from (£1,..., £n) to (a1,...,%n-1). 


2. Take one tetrad type and rotate it 180° to get another tetrad type; these two types 
clearly tile the plane (without further rotations), by repeating a 2 x 2 pattern. 


3. Consider the set of tetrad types X for all positive integers j. The right half 


plane can be tiled in uncountably many ways; but whatever square is placed in the 
center of the plane puts a finite limit on the distance it can be continued to the left. 


4. Systematically enumerate all possible ways to tile an n x n block, for n = 1, 2,..., 
looking for toroidal solutions within these blocks. If there is no way to tile the plane, the 
infinity lemma tells us there is an n with no n x n solutions. If there is a way to tile the 
plane, the assumption tells us that there is an n with an n xn solution containing a rect- 
angle that yields a toroidal solution. Hence in either case the algorithm will terminate. 

[But the stated assumption is false, as shown in the next exercise; and in fact there 
is no algorithm that will determine in a finite number of steps whether or not there 
exists a way to tile the plane with a given set of types. On the other hand, if such a 
tiling does exist, there is always a tiling that is quasitoroidal, in the sense that each of 
its n x n blocks occurs at least once in every f(n) x f(n) block, for some function f. 
See B. Durand, Theoretical Computer Science 221 (1999), 61-75.] 


5. Start by noticing that we need classes 3 a replicated in 2x2 groups in any solution. 
Then, step 1: Considering just the a squares, show that the pattern ae must be 
replicated in 2 x 2 groups of a squares. Step n > 1: Determine a pattern that must 
appear in a cross-shaped region of height and width 2” — 1. The middle of the crosses 
has the pattern as pile replicated throughout the plane. 

For example, after step 3 we will know the contents of 7 x 7 blocks throughout the 


plane, separated by unit length strips, every eight units. The 7 x 7 blocks that are of 
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class Na in the center have the form 


aa BKQ ab BQP aa BBK ab 
yPJ ôNa yRB ôQK yLJ ôNb yPB 
ac BDS ad BQTY ac BBS ad 


yPQ PJ ~yPXB ôNa ~yRQ RB JRB 


aa BUK ab BDP aa BBK ab 
yTJ ôNc yS B ôDS SJ ôNd yTB 
ac BQS ad BDT ac BBS ad 


The middle column and the middle row is the “cross” just filled in during step 3; the 
other four 3 x 3 squares were filled in after step 2; the squares just to the right and 
below this 7 x 7 square are part of a 15 x 15 cross to be filled in at step 4. 

For a similar construction that leads to a set of only 35 tetrad types having nothing 
but nontoroidal solutions, see R. M. Robinson, Inventiones Math. 12 (1971), 177-209. 
Robinson also exhibits a set of siz squarish shapes that tile the plane only nontoroidally, 
even when rotations and reflections are allowed. In 1974, Roger Penrose discovered a 
set of only two polygons, based on the golden ratio instead of a square grid, that tile the 
plane only aperiodically; this led to a set of only 16 tetrad types with only nontoroidal 
solutions [see B. Griinbaum and G. C. Shephard, Tilings and Patterns (Freeman, 1987), 
Chapters 10-11; Martin Gardner, Penrose Tiles to Trapdoor Ciphers (Freeman, 1989), 
Chapters 1-2]. 


6. Let k and m be fixed. Consider an oriented tree whose vertices each represent, 
for some n, one of the partitions of {1,...,n} into k parts, containing no arithmetic 
progression of length m. A node that partitions {1,...,2-+ 1} is a child of one for 
{1,...,n} if the two partitions agree on {1,...,n}. If there were an infinite path to 
the root we would have a way to divide all integers into k sets with no arithmetic 
progression of length m. Hence, by the infinity lemma and van der Waerden’s theorem, 
this tree is finite. (If k = 2, m = 3, the tree can be rapidly calculated by hand, and the 
least value of N is 9. See Studies in Pure Mathematics, ed. by L. Mirsky (Academic 
Press, 1971), 251-260, for van der Waerden’s interesting account of how the proof of 
his theorem was discovered.) 


7. The positive integers can be partitioned into two sets So and Sı such that neither 
set contains any infinite computable sequence (see exercise 3.5-32). So in particular 
there is no infinite arithmetic progression. Theorem K does not apply because there is 
no way to put partial solutions into a tree with finite degrees at each vertex. 


8. Let a “counterexample sequence” be an infinite sequence of trees that violates 
Kruskal’s theorem, if such sequences exist. Assume that the theorem is false; then let 
Tı be a tree with the smallest possible number of nodes such that Tı can be the first 
tree in a counterexample sequence; if Ti, ..., Tj have been chosen, let Tj; be a tree 
with the smallest possible number of nodes such that Ti, ..., Tj, Tj+1 is the beginning 
of a counterexample sequence. This process defines a counterexample sequence (Tn). 
None of these T’s is just a root. Now, we look at this sequence very carefully: 

(a) Suppose there is a subsequence Tni, Tno, ... for which (Tni), (Ing), --- 
is a counterexample sequence. This is impossible; otherwise Ti, ..., Tn,-1, (Tni), 
\(Iny), ... would be a counterexample sequence, contradicting the definition of Ty,. 
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(b) Because of (a), there are only finitely many j for which I(T}j) cannot be 
embedded in I(T) for any k > j. Therefore by taking ni larger than any such j 
we can find a subsequence for which (Tni) C (Tha) C (Tna) C++: 

(c) Now by the result of exercise 2.3.2-22, r(T,,;) cannot be embedded in r(Tn,) 
for any k > j, else Tn; C Tuy Therefore Ti, ..., Tang- r(Tn,), T(Tnz), --. is a 
counterexample sequence. But this contradicts the definition of Ty, . 

Notes: Kruskal, in Trans. Amer. Math. Soc. 95 (1960), 210-225, actually proved 
a stronger result, using a weaker notion of embedding. His theorem does not follow 
directly from the infinity lemma, although the results are vaguely similar. Indeed, Kőnig 
himself proved a special case of Kruskal’s theorem, showing that there is no infinite 
sequence of pairwise incomparable n-tuples of nonnegative integers, where comparabil- 
ity means that all components of one n-tuple are < the corresponding components of 
the other [Matematikai és Fizikai Lapok 39 (1932), 27-29]. For further developments, 
see J. Combinatorial Theory A13 (1972), 297-305. See also N. Dershowitz, Inf. Proc. 
Letters 9 (1979), 212-215, for applications to termination of algorithms. 


SECTION 2.3.4.4 
kt t 
1. In A(z) = Inz + $ ax In(—) =Inz4 > a“ =Inz4 ee 


k>1 k,t>1 t>1 


2. By differentiation, and equating the coefficients of z”, we obtain the identity 


nany = > > dadün+1—k.- 


k>1 d\k 
Now interchange the order of summation. 


4. (a) A(z) certainly converges at least for |z| < 4, since an is less than the number 
of ordered trees bn—1. Since A(1) is infinite and all coefficients are positive, there is a 
positive number a < 1 such that A(z) converges for |z| < a, and there is a singularity 
at z = a. Let p(z) = A(z)/z; since y(z) > e*”), we see that u(z) = m implies 
z<Inm/m, so ~(z) is bounded and limz_,.— w(z) exists. Thus a < 1, and by Abel’s 
limit theorem a = a - exp(a+ $A(a*) + 4A(a?) +--+). 

b) A(z”), A(z’), ... are analytic for |z| < \/a, and $A(27)+4.A(z9)+- <- converges 
uniformly in a slightly smaller disk. 

c) If OF /Ow = a — 1 Æ 0, the implicit function theorem implies that there is an 
analytic function f(z) in a neighborhood of (a, a/a) such that F(z, f(z)) = 0. But this 
implies f(z) = A(z)/z, contradicting the fact that A(z) is singular at a. 

d) Obvious. 

e) OF /Ow = A(z) — 1 and |A(z)| < A(a) = 1, since the coefficients of A(z) are 
all positive. Hence, as in (c), A(z) is regular at all such points. 

f) Near (a,1/a) we have the identity 0 = 6(z — a) + (a/2)(w — 1/a)? + higher 
order terms, where w = A(z)/z; so w is an analytic function of yz = a here by the 
implicit function theorem. Consequently there is a region |z| < a, minus a cut [a, a1] 
in which A(z) has the stated form. (The minus sign is chosen since a plus sign would 
make the coefficients ultimately negative.) VIA 1/2 
(g) Any function of the stated form has coefficient asymptotically ae ( ). 


Note that (22) , (m) x 
=O|-— ; 
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For further details, and asymptotic values of the number of free trees, see R. Otter, 
Ann. Math. (2) 49 (1948), 583-599. 


Bs = 5 ee aa n>. 


ji t2j2+ =n Jt si 


Therefore 
2C(2) +1- 2z = (1-2) (1 — 27) (1 — z?) .... = exp(C(z) + 4O(z7) +--+). 
We find O(z) = z + 27 +22? + 524 + 122 +3326 + 9027 + 26125 + 76629 +---. When 


n > 1, the number of series-parallel networks with n edges is 2c, [see P. A. MacMahon, 
Proc. London Math. Soc. 22 (1891), 330-339]. 

6. zG(z)? = 2G(z) — 2 — zG(z?); G(z) = 1 + z + 27 + 223 + 324 + 629 + 112° 4 
232” +4628 +982? +---. The function F(z) = 1 — zG(z) satisfies the simpler relation 
F(z?) = 2z + F(z)°. [J. H. M. Wedderburn, Annals of Math. (2) 24 (1922), 121-140.] 


7. Gn= ca™n—3/2(1 + O(1/n)), where c ~ 0.7916031835775, a ~ 2.483253536173. 
8. 


ma NNO Gade 


9. If there are two centroids, by considering a path from one to the other we find that 
there can’t be intermediate points, so any two centroids are adjacent. A tree cannot 
contain three mutually adjacent vertices, so there are at most two. 


10. If X and Y are adjacent, let s(X,Y) be the number of vertices in the Y subtree 
of X. Then s(X,Y) + s(Y,X) = n. The argument in the text shows that if Y is a 
centroid, weight(X) = s(X,Y). Therefore if both X and Y are centroids, weight(X) = 
weight(Y) = n/2. 

In terms of this notation, the argument in the text goes on to show that if 
s(X,Y) > s(Y,X), there is a centroid in the Y subtree of X. So if two free trees 
with m vertices are joined by an edge between X and Y, we obtain a free tree in which 
s(X,Y) = m = s(Y, X), and there must be two centroids (namely X and Y). 

[It is a nice programming exercise to compute s(X,Y) for all adjacent X and Y 
in O(n) steps; from this information we can quickly find the centroid(s). An efficient 
algorithm for centroid location was first given by A. J. Goldman, Transportation Sci. 
5 (1971), 212-221] 

11. zT(z)’ = T(z) —1; thus z + T(z)’ = T(z)'"*. By Eq. 1.2.9-(21), T(z) = 
Xn An(1, —t)2", so the number of t-ary trees is 


( 1+ mm o ‘Ge 1 
n 1+tn n/(t-1)n4+1° 
12. Consider the directed graph that has one arc from V; to V; for all i # j. The 


matrix Ao of exercise 2.3.4.2—19 is a combinatorial (n — 1) x (n — 1) matrix with n — 1 
on the diagonal and —1 off the diagonal. So its determinant is 


(n+ (n— iain =n", 


the number of oriented trees with a given root. (Exercise 2.3.4.2—20 could also be used.) 
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13. 


14. True, since the root will not become a leaf until all other branches have been 
removed. 


15. In the canonical representation, Vi, V2,..., Va—1, f(Vn—-1) is a topological sort of 
the oriented tree considered as a directed graph, but this order would not in general be 
output by Algorithm 2.2.3T. Algorithm 2.2.3T can be changed so that it determines 
the values of Vi, V2, ..., Vn—1 if the “insert into queue” operation of step T6 is replaced 
by a procedure that adjusts links so that the entries of the list appear in ascending 
order from front to rear; then the queue becomes a priority queue. 

(However, a general priority queue isn’t needed to find the canonical represen- 
tation; we only need to sweep through the vertices from 1 to n, looking for leaves, 
while pruning off paths from new leaves less than the sweep pointer; see the following 
exercise.) 


16. D1. Set C[1] + --- + C[n] + 0, then set C[f(V;)] + CEf(V;)] +1 for1 <j <n. 
(Thus vertex k is a leaf if and only if C[k] = 0.) Set k + 0 and j + 1. 


D2. Increase k one or more times until C[k] = 0, then set 1 + k. 

D3. Set PARENT[] + f(V;), l4 f(V;), CEU 4+ CE] — 1, and j + j +1. 
D4. If j = n, set PARENT[/] + 0 and terminate the algorithm. 

D5. If Cl] = 0 and l < k, go to D3; otherwise go back to D2. J 


17. There must be exactly one cycle £1, 2,..., Ee where f(x;) = £j+1 and f (£p) = z1. 
We will enumerate all f having a cycle of length k such that the iterates of each 
x ultimately come into this cycle. Define the canonical representation f(Vi), f(V2), 
.-, f(Vm—x) as in the text; now f(Vm-x) is in the cycle, so we continue to get a 
“canonical representation” by writing down the rest 
of the cycle f(f(Vm-_—x)), f(f(f(Vm-—x))), ete. For 
example, the function with m = 13 whose graph is 
shown here leads to the representation 3, 1, 8, 8, 
1, 12, 12, 2, 3, 4, 5, 1. We obtain a sequence of 
m — 1 numbers in which the last k are distinct. Con- 
versely, from any such sequence we can reverse the 
construction (assuming that k is known); hence there 
are precisely m£m™—*—' such functions having a k- 
cycle. (For related results, see exercise 3.1-14. The 
formula m™”~'Q(m) was first obtained by L. Katz, 
Annals of Math. Statistics 26 (1955), 512-517.) 
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18. To reconstruct the tree from a sequence s1, S2, ..., Sn—1, begin with sı as the 
root and successively attach arcs to the tree that point to s1, S2, ...; if vertex są has 
appeared earlier, leave the initial vertex of the arc leading to s,_1 nameless, otherwise 
give this vertex the name s,. After all n — 1 arcs 
have been placed, give names to all vertices that re- 3% 
main nameless by using the numbers that have not 
yet appeared, assigning names in increasing order 
to nameless vertices in the order of their creation. 
For example, from 3, 1, 4, 1, 5, 9, 2, 6, 5 we 
would construct the tree shown on the right. There 
is no simple connection between this method and 
the one in the text. Several more representations 
are possible; see the article by E. H. Neville, Proc. 7è 9 
Cambridge Phil. Soc. 49 (1953), 381-385. 


19. The canonical representation will have precisely n — k different values, so we 
enumerate the sequences of n—1 numbers with this property. The answer is n°— {oF ; 


1 
4g p 


eco 


2 6 
< < 


20. Consider the canonical representation of such trees. We are asking how many 
terms of (a1 ++- + £n)” 7} have ko exponents zero, kı exponents one, etc. This is 
plainly the coefficient of such a term times the number of such terms, namely 


(n—1)! . n! 
(O!)Fo(1!)*1 (nl)  kol kil... kn!” 


21. There are none with 2m vertices; if there are n = 2m + 1 vertices, the answer is 
obtained from exercise 20 with ko = m+ 1, k2 = m, namely ?™**)(2m)!/2™. 

22. Exactly n”~?; for if X is a particular vertex, the free trees are in one-to-one 
correspondence with oriented trees having root X. 


23. It is possible to put the labels on every unlabeled, ordered tree in n! ways, and 
each of these labeled, ordered trees is distinct. So the total number is n!bn-ı = 
(2n — 2)!/(n — 1)!. 

24. There are as many with one given root as with another, so the answer in general 
is 1/n times the answer in exercise 23; and in this particular case the answer is 30. 


25. For 0 <q <n, r(n,q) = (n — q)n1*. (The special case s = 1 in Eq. (24).) 


26. (k=7) 8AYcllow 9eBlue 
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27. Given a function g from {1,2,...,r} to {1,2,...,q} such that adding arcs from V;, 
to U(x) introduces no oriented cycles, construct a sequence ai, ..., ar as follows: Call 
vertex Vp “free” if there is no oriented path from V; to Vk for any j 4 k. Since there 
are no oriented cycles, there must be at least one free vertex. Let bı be the smallest 
integer for which Vp, is free; and assuming that bi, ..., b; have been chosen, let bi+1 
be the smallest integer different from bi, ..., be for which Vi, 41 18 free in the graph 
obtained by deleting the arcs from V», to Uge) for 1 < k < t. This rule defines a 
permutation bi b2...b, of the integers {1,2,...,r}. Let ap = g(bp) for 1 < k < r; this 
defines a sequence such that 1 < ax < q for 1 < k < r, and 1 < ar <p. 

Conversely if such a sequence a1, ..., ar is given, call a vertex Vp “free” if there 
is no j for which aj > p and f(aj) = k. Since ar < p there are at most r — 1 non- 
free vertices. Let bı be the smallest integer for which Vp, is free; and assuming that 
bi, ..., 6 have been chosen, let b++ı be the smallest integer different from bi, ..., bt 
for which Vj,,, is free with respect to the sequence at+1, ..., ar. This rule defines a 
permutation bib2...b, of the integers {1,2,...,r}. Let g(b,) = ax for 1 < k < r; this 
defines a function such that adding arcs from Vz to U,(,) introduces no oriented cycles. 


28. Let f be any of the n™~* functions from {2,...,m} to {1,2,...,n}, and consider 
the directed graph with vertices U1, ..., Um, Vi, ..., Vn and arcs from Ux to V(x) for 
1<k<m. Apply exercise 27 with p = 1, q = m, r = n, to show that there are m"~! 
ways to add further arcs from the V’s to the U’s to obtain an oriented tree with root 
Uı. Since there is a one-to-one correspondence between the desired set of free trees and 
the set of oriented trees with root U1, the answer is nm tmnt, [This construction can 
be extensively generalized; see D. E. Knuth, Canadian J. Math. 20 (1968), 1077—1086.| 


29. If y = 2", then (tz)y = lny, and we see that it is sufficient to prove the identity 
for t = 1. Now if zx = Ing we know by exercise 25 that x” = >, E;(m,1)z* for 
nonnegative integers m. Hence 


reer ae Dee eae 


k i j,k 


pe. a Cy eei 


k 


[Exercise 4.7-22 derives considerably more general results.] 


30. Each graph described defines a set Cs C {1,..., n}, where j is in Cz if and only 
if there is a path from t; to ri for some i < x. For a given Cz each graph described 
is composed of two independent parts: one of the x(a + e121 +- + enan) r tent 
graphs on the vertices ri, Sjk, tj for i < x and j € Cz, where e; = [j € Cz], plus one of 


the y(yt+ (l—e1)zi+---+(1 enen) et graphs on the other vertices. 
31. G(z) = z + G(z)? + G(z)? + G(2)f +.) = z + G(z)?/(1 — G(z)). Hence G(z) = 
t(1+z 1—6z+27) = 24+ 274 322 +1124 4 452° +--+. [Notes: Another 


problem equivalent to this one was posed and solved by E. Schréder, Zeitschrift fiir 
Mathematik und Physik 15 (1870), 361-376, who determined the number of ways to 
insert nonoverlapping diagonals in a convex (n + 1)-gon. These numbers for n > 1 
are just half the values obtained in exercise 2.2.1-11, since Pratt’s grammar allows 
the root node of the associated parse tree to have degree one. The asymptotic value 
is calculated in exercise 2.2.1-12. Curiously, the value [z’°]G(z) = 103049 seems 
to have been calculated already by Hipparchus in the second century B.C., as the 
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number of “affirmative compound propositions that can be made from only ten simple 
propositions”; see R. P. Stanley, AMM 104 (1997), 344-350; F. Acerbi, Archive for 
History of Exact Sciences 57 (2003), 465-502.] 


32. Zero if no Æ 1 + n2 + 2na + 3n4+--- (see exercise 2.3-21), otherwise 


(no + ni +: +m —1)!/no! nil... Nml. 


To prove this result we recall that an unlabeled tree with n = no + nı +--+: +m 
nodes is characterized by the sequence dı dz2...dn of the degrees of the nodes in 
postorder (Section 2.3.3). Furthermore such a sequence of degrees corresponds to a 
tree if and only if Djal — dj) > 0 for 0 < k < n. (This important property of Polish 
postfix notation is readily proved by induction; see Algorithm 2.3.3F with f a function 
that creates a tree, like the TREE function of Section 2.3.2.) In particular, dı must 
be 0. The answer to our problem is therefore the number of sequences dz2...dn with 
nj occurrences of j for j > 0, namely the multinomial coefficient 


n—-1 
no— l1, ni,...;, Nm] 


minus the number of such sequences də...dn for which D — dj) < 0 for some 
k>2. 

We may enumerate the latter sequences as follows: Let t be minimal such that 
Sal! — dj) < 0; then Xa — dj) = —s where 1 < s < d+, and we may form the 
subsequence d3... dh, = dy_1... d20dt+1... dn, which has nj occurrences of j for j # dt, 
nj — 1 occurrences of j for j = d+. Now pea — d}) is equal to d when k = n, and 
equal to d — s when k = t; when k < t, it is 


Y a-d- X (-d)< X -d)=ad-s-1. 
2<j<t 2<j<t—k 2<j<t 


It follows that, given s and any sequence d...d/),, the construction can be reversed; 
hence the number of sequences dz2...dn that have a given value of d; and s is the 
multinomial coefficient i 
n= 
re S 


The number of sequences də2...dn that correspond to trees is therefore obtained by 
summing over the possible values of d: and s: 


and the latter sum is 1. 

An even simpler proof of this result has been given by G. N. Raney (Transactions of 
the American Math. Society 94 (1960), 441-451). If dı dz...dy is any sequence with 
nj appearances of j, there is precisely one cyclic rearrangement dz ...dndi...dk—1 
that corresponds to a tree, namely the rearrangement where k is maximal such that 
2e (1 — d;) is minimal. [This argument in the case of binary trees was apparently 
first discovered by C. S. Peirce in an unpublished manuscript; see his New Elements of 
Mathematics 4 (The Hague: Mouton, 1976), 303-304. It was discovered in the case of 
t-ary trees by Dvoretzky and Motzkin, Duke Math. J. 14 (1947), 305-313.] 

Still another proof, by G. Bergman, inductively replaces dkdk+1ı by (dk +dk+1— 1) 
if dp > 0 [Algebra Universalis 8 (1978), 129-130]. 
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The methods above can be generalized to show that the number of (ordered, 


unlabeled) forests having f trees and n; nodes of degree j is (n — 1)! f/no!ni!...mm!, 
provided that the condition no = f + n2 + 2n3 +--- is satisfied. 

33. Consider the number of trees with ni nodes labeled 1, n2 nodes labeled 2, ..., and 
such that each node labeled j has degree ej. Let this number be c(n1, n2,...), with the 
specified degrees e1, e2, ... regarded as fixed. The generating function G(z1, z2,...) = 


Yo c(i, n2,...) 271257... satisfies the identity G = 2G! +---+ 2-G®", since 2;G° 
enumerates the trees whose root is labeled j. And by the result of the previous exercise, 
(ny +n2+---—1)! 

C(n,N2,..-) = ni! nel... 
0, otherwise. 


, if (1—e1)ni + (1 —e2)ne4+---=1; 


More generally, since Gf enumerates the number of ordered forests having such 
labels, we have for integer f > 0 


(nr +n2+----1)f n n 
wf = 5 ninal Bir Bor Saui 
f=(1—e1)nı+(1—e2)n2 +: a 


These formulas are meaningful when r = oo, and they are essentially equivalent to 
Lagrange’s inversion formula. 


SECTION 2.3.4.5 


1. There are (3) in all, since the nodes numbered 8, 9, 10, 11, 12 may be attached in 
any of eight positions below 4, 5, 6, and 7. 


2. 


-li = 1, we 


3. By induction on m, the condition is necessary. Conversely if 4 2 
want to construct an extended binary tree with path lengths li, ..., lm. When m = 1, 
we have lı = 0 and the construction is trivial. Otherwise we may assume that the l’s 
are ordered so that lı = l2 = --- = la > lq+1 È lq+2 > ++: = lm > O for some q with 
1<q< m. Now 227-1 = Dei 21=lj=1 — iq + integer, hence q is even. By induction 
on m there is a tree with path lengths lı — 1, l3, lu, ..., lm; take such a tree and 
replace one of the external nodes at level lı — 1 by an internal node whose children are 


at level l = lə. 
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4. First, find a tree by Huffman’s method. If wj < w 41, then lj > lj+1, since the 
tree is optimal. The construction in the answer to exercise 3 now gives us another 
tree with these same path lengths and with the weights in the proper sequence. For 
example, the tree (11) becomes 


2 3 Reference: CACM 7 (1964), 166-169. 
5. (a) bnp = 5 bkrbis. Hence zB(w, wz)? = B(w, z) — 1. 


k+l=n—1 
r+s+n—1=p 


(b) Take the partial derivative with respect to w: 
2zB(w, wz)(Bu(w, wz) + 2Bz(w, wz)) = Bu (w, z). 


Therefore if H(z) = Bu(1,z) = >>, hnz”, we find H(z) = 2zB(z)(H(z)+ z2B'(z)); and 
the known formula for B(z) implies 


1 tf tz a 3n+1/2n 
H = 1 hn = 4" — . 
Ca eee + (= k 5o zar 


The average value is hn/bn. (c) Asymptotically, this comes to nyrn — 3n + O(vn). 

For the solution to similar problems, see John Riordan, IBM J. Res. and Devel. 4 
(1960), 473-478; A. Rényi and G. Szekeres, J. Australian Math. Soc. 7 (1967), 497-507; 
John Riordan and N. J. A. Sloane, J. Australian Math. Soc. 10 (1969), 278-282; and 
exercise 2.3.1-11. 

6. n+s—1=tn. 

7. E=(t—1)I+tn. 

8. Summation by parts gives )77_, [log,((t — 1)k)| = nq — Ð k, where the sum on 
the right is over values of k such that 0 < k < n and (t—1)k+1=t#? for some j. The 
latter sum may be rewritten wa’ —1)/(t-1). 

9. Induction on the size of the tree. 

10. By adding extra zero weights, if neces- 
sary, we may assume that m mod (t — 1) = 1. 
To obtain a t-ary tree with minimum weighted 
path length, combine the smallest t values at 
each step and replace them by their sum. The 
proof is essentially the same as the binary case. 
The desired ternary tree is shown. 

F. K. Hwang has observed [SIAM J. Appl. 
Math. 37 (1979), 124-127] that a similar pro- 
cedure is valid for minimum weighted path 
length trees having any prescribed multiset of 
degrees: Combine the smallest t weights at 
each step, where t is as small as possible. 
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11. The “Dewey” notation is the binary representation of the node number. 


12. By exercise 9, it is the internal path length divided by n, plus 1. (This result holds 
for general trees as well as binary trees.) 


13. [See J. van Leeuwen, Proc. 3rd International Colloq. Automata, Languages and 
Programming (Edinburgh University Press, 1976), 382—410.] 


H1. [Initialize.| Set A[m — 1 + i] + wi for 1 < i < m. Then set A[2m] + oo, 
z|} m, i} m+1, j4 m-1, k4 m. (During this ner Ali] <- < 
A[2m — 1] is the queue of unused external weights; A[k] > --- > A[j] is Aie 
queue of unused internal weights, empty if j < k; the current left and right 
pointers are x and y.) 

H2. [Find right pointer.] If j < k or A[i] < A[j], set y + i andi + i+1; otherwise 
set y + jandj¢j-l. 

H3. [Create internal node.] Set k + k—1, L[k] + z, Rik] + y, A[k] + Ala]+ Aly]. 

H4. [Done?] Terminate the algorithm if k = 1. 


H5. [Find left pointer.] (At this point j > k and the queues contain a total of k 
unused weights. If Aly] < 0 we have j = k, i = y + 1, and Afi] > A[j].) If 
Ali] < Aly], set x + i and i + i+ 1; otherwise set x + j and j + j —1. 
Return to step H2. I 
14. The proof for k = m — 1 applies with little change. [See SIAM J. Appl. Math. 21 
(1971), 518.] 


15. Use the combined-weight functions (a) 1 + max(wi,w2) and (b) zwı + xwe, re- 
spectively, instead of wı + w2 in (9). [Part (a) is due to M. C. Golumbic, IEEE Trans. 
C-25 (1976), 1164-1167; part (b) to T. C. Hu, D. Kleitman, and J. K. Tamaki, SIAM 
J. Appl. Math. 37 (1979), 246-256. Huffman’s problem is the limiting case of (b) as 
z> 1, since D(1 + e) w = iw; +e wlj + O(e*).] 

D. Stott Parker, Jr., has pointed out that a Huffman-like algorithm will also find 
the minimum of wiz! + +--+ wma!” when 0 < 2 < 1, if the two maximum ae 
are combined at each ou as in part (b). In particular, the minimum of w127" +--+ 
Wm2'™, when wi < ++ < wm, is wi/2 +--+- + Wm—-1/27 71 + wm /2™ 1. See D. E. 
Knuth, J. Comb. Theory A32 (1982), 216-224, for further generalizations. 


16. Let lm41 =Un41 =0. Then 


m m m k m k m 
2" < So wilh = h- ha) So w < YO- ea) Sw = SO wh, 


since Ui > Ui 41 as in exercise 4. The same proof holds for many other kinds of optimum 
trees, including those of exercise 10. 


17. (a) This is exercise 14. (b) We can extend f(n) to a concave function f(x), so the 
stated inequality holds. Now F(m) is the minimum of Da f(sj), where the s; are 
internal node weights of an extended binary tree on the weights 1, 1, ..., 1. Huffman’s 
algorithm, which constructs the complete binary tree with m — 1 internal nodes in this 
case, yields the optimum tree. The choice k = 208(2/3)] defines a binary tree with the 
same internal weights, so it yields the minimum in the recurrence, for each n. [SIAM J. 
Appl. Math. 31 (1976), 368-378.] We can evaluate F (n) in O(log n) steps; see exercises 
5.2.3-20 and 5.2.3-21. If f(n) is convex instead of concave, so that A? f(n) > 0, the 
solution to the recurrence is obtained when k = |n/2]. 
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SECTION 2.3.4.6 

1. Choose one edge of the polygon and call it the base. Given a triangulation, let the 
triangle on the base correspond to the root of a binary tree, and let the other two sides 
of that triangle define bases of left and right subpolygons, which correspond to left 
and right subtrees in the same way. We proceed recursively until reaching “2-sided” 
polygons, which correspond to empty binary trees. 

Stating this correspondence another way, we can label the non-base edges of a 
triangulated polygon with the integers 0, ..., n; and when two adjacent sides of a 
triangle are labeled a and £ in clockwise order, we can label the third side (a3). The 
label of the base then characterizes the binary tree and the triangulation. For example, 


ae 
A 5 
(45) 
% 4d) 6 
A 
G 
N 
N A a 
i 
Lo 
Ea 2 
ay S 
A G = 
O, Ox 
A A Ao Op 
e 2, D 
0 © 


(((01)2)((3(45))((67)(89)))) 


corresponds to the binary tree shown in 2.3.1-(1). [See H. G. Forder, Mathematical 
Gazette 45 (1961), 199-201.] 

2. (a) Take a base edge as in exercise 1, and give it d descendants if that edge is part 
of a (d+ 1)-gon in the dissected r-gon. The other d edges are then bases for subtrees. 
This defines a correspondence between Kirkman’s problem and all ordered trees with 
r — 1 leaves and k + 1 nonleaves, having no nodes of degree 1. (When k = r — 3 we 


have the situation of exercise 1.) 

(b) There are Cy) (7°) sequences did2...d,+ of nonnegative integers such that 
r — 1 of the d’s are 0, none of them are 1, and the sum is r+ k — 1. Exactly one of 
the cyclic permutations didz2...dr+x, d2...dr+kdi, ..., dr+edi...dr+k—1 satisfies the 
additional property that )77_,(1—dj;) > 0 forl<q<rtk. 

[Kirkman gave evidence for his conjecture in Philos. Trans. 147 (1857), 217-272, 
§22. Cayley proved it in Proc. London Math. Soc. 22 (1891), 237-262, without noticing 
the connection to trees.] 

3. (a) Let the vertices be {1,2,...,n}. Draw an RLINK from i to j if i and j are 
consecutive elements of the same part and i < j; draw an LLINK from j to j + 1 if 
j+ 1 is the smallest of its part. Then there are k — 1 nonnull LLINKs, n — k nonnull 
RLINKs, and we have a binary tree whose nodes are 12...n in preorder. Using the 
natural correspondence of Section 2.3.2, this rule defines a one-to-one correspondence 
between “partitions of an n-gon’s vertices into k noncrossing parts” and “forests with 
n vertices and n — k + 1 leaves.” Interchanging LLINK with RLINK also gives “forests 
with n vertices and k leaves.” 

(b) A forest with n vertices and k leaves also corresponds to a sequence of nested 
parentheses, containing n left parentheses, n right parentheses, and k occurrences of 
“()”. We can enumerate such sequences as follows: 
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Say that a string of Os and 1s is an (m,n, k) string if there are m 0s, n 1s, and 
k occurrences of “01”. Then 0010101001110 is a (7,6, 4) string. The number of (m, n, k) 
strings is (7) G) because we are free to choose which Os and 1s will form the 01 pairs. 

Let S(a) be the number of Os in a minus the number of 1s. We say that a string o 
is good if S(a) > 0 whenever a is a prefix of o (in other words, if o = aß implies that 
S(a) > 0); otherwise ø is bad. The following alternative to the “reflection principle” of 
exercise 2.2.1—4 establishes a one-to-one correspondence between bad (n,n, k) strings 
and arbitrary (n — 1,n + 1, k) strings: 

Any bad (n,n, k) string o can be written uniquely in the form o = a08, where 
aF and 6 are good. (Here a is the string obtained from a by reversing it and 
complementing all the bits.) Then o’ = a1 is an (n — 1,n + 1, k) string. Conversely, 
every (n — 1,n + 1, k) string can be written uniquely in the form a1 where a? and 8 
are good, and a0£ is then a bad (n,n, k) string. 

Thus the number of forests with n vertices and k leaves is (2) (œ) - C) C4 = 


k) \k k k 
(Ra) (2) - CS (74) =n! (n—1)!/(n—k +1)! (n—k)! k! (k—1)!, a so-called Narayana 
number |T. V. Narayana, Comptes Rendus Acad. Sci. 240 (Paris, 1955), 1188-1189]. 

Notes: G. Kreweras, Discrete Math. 1 (1972), 333-350, enumerated noncrossing 
partitions in a different way. The partial ordering of partitions by refinement leads to an 
interesting partial ordering of forests, different from the one discussed in exercise 2.3.3— 
19; see Y. Poupard, Cahiers du Bureau Univ. de Recherche Opér. 16 (1971), Chapter 8; 
Discrete Math. 2 (1972), 279-288; P. Edelman, Discrete Math. 31 (1980), 171-180, 40 
(1982), 171-179; N. Dershowitz and S. Zaks, Discrete Math. 64 (1986), 215-218. 

A third way to define a natural lattice ordering of forests was introduced by 
R. Stanley in Fibonacci Quarterly 13 (1975), 215-232: Suppose we represent a forest 
by a string o of Os and 1s representing left and right parentheses as above; then ø < a’ 
if and only if S(o,) < S(o;,) for all k, where oj, denotes the first k bits of ø. Stanley’s 
lattice is distributive, unlike the other two. 


4. Let m = n+ 2; by exercise 1, we want a correspondence between triangulated 
m-gons and (m — 1)-rowed friezes. First let’s look more closely at the previous cor- 
respondence, by giving a “top-down” labeling to the edges of a triangulation instead 
of the “bottom-up” one considered earlier: Assign the empty label e to the base, then 
recursively give the labels aL and aR to the opposite edges of a triangle whose base is 
labeled a. For example, the previous diagram becomes 


pIe e te 
S a e 
Q R <o 
aN 
gas 
z E 
5 gs 
eh 
2 2 
A SE 
> 
ey 
SS 
22> Lose 


€ 


under these new conventions. If the base edge in this example is called 10, while the 
other edges are 0 to 9 as before, we can write 0 = 10LLL, 1 = 10LLR, 2 = 10LR, 
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3 = 10RLL, etc. Any of the other edges can also be chosen as the base; thus, if 0 is 
chosen we have 1 = 0L, 2 = ORL, 3 =ORRLLL, etc. It is not difficult to verify that 
if u = va we have v = ua‘, where a” is obtained by reading a from right to left and 
interchanging L with R. For example, 10 = ORRR = 1LRR = 2LR = 3RRL, etc. If 
u, v, and w are edges of the polygon with w = uaLy and w = vß Ry, then u = vBLa™ 
and v = ua Rp”. 

Given a triangulation of a polygon whose edges are numbered 0, 1, ..., m — 1, we 
define (u, v) for any pair of distinct edges u and v as follows: Let u = va, and interpret 
a as a 2 x 2 matrix by letting L = (} |) and R = ({ 9). Then (u,v) is defined to 
be the element in the upper left corner of a. Notice that a7 is the transpose of the 
matrix a, since R = L”; hence we have (v,u) = (u,v). Notice also that (u,v) = 1 if 
and only if w- and v_ are joined by an edge of the triangulation, where u_ denotes 
the vertex between edges u and u — 1. 

Let (u, u) = 0 for all polygon edges u. We can now prove that v = wa implies 


hs ( (wo) 5 eee) ) for all u # v, (*) 


(u+1,v) (u+1,v+ 1) 


where u+ 1 and v+ 1 are the clockwise successors of u and v. The proof is by induction 
on m: Eq. (x) is trivial when m = 2, since the two parallel edges u and v are then 
related by u = ve, and a = e is the identity matrix. If any triangulation is augmented 
by extending some edge v with a triangle vv’ v”, then v = ua implies v’ = uaL and 
v” = uaR; hence (u, v’) and (u, v”) in the extended polygon are respectively equal to 


(u,v) and (u,v) + (u,v + 1) in the original one. It follows that 


_( x") (u, v” +1) 
a] and aR = a (u+1,v” +1) i 


and (*) remains true in the extended polygon. 
The frieze pattern corresponding to the given triangulation is now defined to be 
the periodic sequence 


(0,1) (1,2) (2,3) ue (m—1,0) (0,1) (1,2) 
(0,2) (1,3) (2,4) (m—1,1) (0,2) (1,3) 

(m—1,2) (0,3) (1,4) eda (m—2,1) (m—1,2) (0,3) 
(m—1,3) (0,4) (1,5) (m—2,2) (m—1,3) (0,4) 


and so on until m — 1 rows have been defined; the final row begins with ([m/2] + 1, 
[m/2]) when m > 3. Condition (x) proves that this pattern is a frieze, namely that 


(u,v)(u + 1,v + 1) — (u,v + 1)(u+1,v)=1, (2) 


because det L = det R = 1 implies deta = 1. Our example triangulation yields 


Ved, Ae We De ed BT De De Te Be De oe abe? Te Ds 
1242 15 13 1 43 «12 42 «215 13 «21 «4 
217 71 44 2 2 3 «i112 «1 «7 «7 «21~«-4«<4 2? «2 8 
1 3 12 3 3 3 7 1 5 8 7 13 12 3 3 3 7 1 5 8 
3 2 5 5 8 2 5 3 2135 3 2 5 5 8 2 5 3 2 13 
5 3 2 135 3 2 5 5 8 2 5 3 2 135 3 2 5 5 8 
3 7 1 5 8 7 1 312 3 3 3 7 «15 8 7 1 38 12 3 
4 2 2 3 11 2 1 7 7 1 4 4 2 2 3 112 1 7 7 1 
5 1 3 1 4 3 12 42 1 5 «13 1 43 «212 4 2 
1 PORE O ds cle P O a O dis al D de te C a le sel 
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The relation (u,v) = 1 defines the edges of the triangulation, hence different triangu- 
lations yield different friezes. To complete the proof of one-to-one correspondence, we 
must show that every (m — 1)-rowed frieze pattern of positive integers is obtained in 
this way from some triangulation. 

Given any frieze of m — 1 rows, extend it by putting a new row 0 at the top and 
a new row m at the bottom, both consisting entirely of zeros. Now let the elements of 
row 0 be called (0,0), (1,1), (2,2), etc., and for all nonnegative integers u < v <u+tm 
let (u, v) be the element in the diagonal southeast of (u, u) and in the diagonal southwest 
of (v,v). By assumption, condition (xx) holds for all u < v < u + m. We can in fact 
extend (**) to the considerably more general relation 


(t, w)(v, w) + (t, w)(u, v) = (t, v)(u, w) fort<u<vu<w<tt+m. (2k) 


For if (xxx) is false, let (t,u,v,w) be a counterexample with the smallest value of 
(w—t)m+u-—t+w-—v. Clearly t # u and v # w. Case 1: t+1 <u. Then (***) 
holds for (t,t + 1,v,w), (t,t + 1,u,v), and (t + 1,u,v,w), so we find ((t,u)(v, w) + 
(t, w)(u, v))(t+1,v) = (t,v)(u, w)(t +1, v); this implies (t + 1, v) = 0, a contradiction. 
Case 2: v+1 < w. Then (xxx) holds for (t, u, w—1, w), (u,v, w—1, w), and (t, u, v, w—1); 
we obtain a similar contradiction (u, w — 1) = 0. Case 3: u =t+1 and w=v+1. In 
this case (xxx) reduces to (**). 

Now we set u=t+1 and w = t + m in (*«x), obtaining (t,v) = (v,t + m) for 
t<v<t+m, because (t+ 1,t +m) = 1 and (t,t+m) = 0. We conclude that the 
entries of any (m — 1)-rowed frieze are periodic: (u,v) = (v, u+ m) = (u+ m, v +m) = 
(v+tm,u+2m)=---. 

Every frieze pattern of positive integers contains a 1 in row 2. For if we set t = 0, 
v = u + 1, and w = u + 2 in (xxx) we get (0,u + 1)(u,u + 2) = (0,u) + (0,u + 2), 
hence (0,u + 2) — (0,u + 1) > (0,u + 1) — (0,u) if and only if (u,u + 2) > 2. This 
cannot hold for all u in the range 0 < u < m — 2, because (0,1) — (0,0) = 1 and 
(0,m) — (0,m — 1) = —1. 

Finally, if m > 3 we cannot have two consecutive 1s in row 2, because (u, u + 2) = 
(u+1,u +3) = 1 implies (u, u +3) = 0. Therefore we can reduce the frieze to another 
one with m reduced by 1, as illustrated here for 7 rows reduced to 6: 


ali, k aL GL, “sl, <2 e - 
a b c d+1 1 e+l y z 
p q cr d e uyv w 
uqtur s u qur s 
uty v w p q cr d eœ ; 
y z a b c dtl 1 e+l1... 
Lo al ale > LE qed: -a 


The reduced frieze corresponds to a triangulation, by induction, and the unreduced 
frieze corresponds to attaching one more triangle. [Math. Gazette 57 (1974), 87-94, 
175-183; Conway and Guy, The Book of Numbers (New York: Copernicus, 1996), 
74-76, 96-97, 101-102.] 

Notes: This proof demonstrates that the function (u,v), which we defined on any 
triangulation via 2 x 2 matrices, satisfies (xxx) whenever (t, u,v, w) are edges of the 
polygon in clockwise order. We can express each (u,v) as a polynomial in the numbers 
aj = (j — 1,7 + 1); these polynomials are essentially identical to the “continuants” 
discussed in Section 4.5.3, except for the signs of individual terms. In fact, (j,k) = 
i Eti Ky 5-1 (iaj41, iaj42,---,ia,-1). Thus (***) is equivalent to Euler’s identity 
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for continuants in the answer to exercise 4.5.3-32. The matrices L and R have the 
interesting property that any 2 x 2 matrix of nonnegative integers with determinant 1 
can be expressed uniquely as a product of L’s and R’s. 

Many other interesting relationships are present; for example, the numbers in 
row 2 of an integer frieze count the number of triangles touching each vertex of the 
corresponding triangulated polygon. The total number of occurrences of (u,v) = 1 in 
the basic region 0 < u < v— 1 < m — 1 and (u,v) ¥ (0,m — 1) is the number of 
diagonals (chords) of the triangulation, namely m — 3 = n — 1. The total number of 2s 
is also n — 1, because (u,v) = 2 if and only if u- and v_ are opposing vertices of the 
two triangles adjacent to a chord. 

Another interpretation of (u,v) was found by D. Broline, D. W. Crowe, and I. M. 
Isaacs [Geometriz Dedicata 3 (1974), 171-176]: It is the number of ways to match the 
v — u — 1 vertices between edges u and v — 1 with distinct triangles adjacent to those 
vertices. 


SECTION 2.3.5 


1. A List structure is a directed graph in which the arcs leaving each vertex are 
ordered, and where some of the vertices that have out-degree 0 are designated “atoms.” 
Furthermore there is a vertex S such that there is an oriented path from S to V for all 
vertices V # S. (With directions of arcs reversed, S would be a “root.”) 


2. Not in the same way, since thread links in the usual representation lead back to 
“PARENT,” which is not unique for sub-Lists. The representation discussed in exercise 
2.3.4.2-25, or some similar method, could perhaps be used (but this idea has not yet 
been exploited at the time of writing). 


3. As mentioned in the text, we prove also that P = PO upon termination. If only 
PO is to be marked, the algorithm certainly operates correctly. If n > 1 nodes are 
to be marked, we must have ATOM(PO) = 0. Step E4 then sets ALINK (P0) + A and 
executes the algorithm with PO replaced by ALINK(PO) and T replaced by PO. By 
induction (note that since MARK(PO) is now 1, all links to PO are equivalent to A by 
steps E4 and E5), we see that ultimately we will mark all nodes on paths that start with 
ALINK(PO) and do not pass through PO; and we will then get to step E6 with T = PO and 
P = ALINK(PO). Now since ATOM(T) = 1, step E6 restores ALINK(PO) and ATOM(PO) 
and we reach step E5. Step E5 sets BLINK(PO) + A, etc., and a similar argument 
shows that we will ultimately mark all nodes on paths that start with BLINK (PO) and 
do not pass through PO or nodes reachable from ALINK(P0O). Then we will get to E6 
with T = PO, P = BLINK(PO), and finally we get to E6 with T = A, P = PO. 


4. The program that follows incorporates the suggested improvements in the speed 
of processing atoms that appear in the text after the statement of Algorithm E. 

In steps E4 and E5 of the algorithm, we want to test if MARK(Q) = 0. If NODE(Q) = 
+0, this is an unusual case that can be handled properly by setting it to —0 and treating 
it as if it were originally —0, since it has ALINK and BLINK both A. This simplification 
is not reflected in the timing calculations below. 

rll =P, rl2 =T, rI3 = Q, and rX = —1 (for setting MARKs). 


01 MARK EQU 
02 ATOM EQU 
03 ALINK EQU 
04 BLINK EQU 


e NBe O 
awe o 
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05 E1 LD1 PO 1 El. Initialize. P < PO. 

06 ENT2 0 1 TA. 

07 ENTX -1 1 rx + —1l. 

08 E2 STX 0,1(MARK) 1 E2. Mark. MARK(P) < 1. 

09 E3 LDA 0,1(ATOM) 1 E3. Atom? 

10 JAZ E4 1 Jump if ATOM(P) = 0. 

11 E6 J2Z DONE n E6. Up. 

12 ENT3 0,2 n—-1 Qer. 

13 LDA 0,3(ATOM) n-—1 

{2 JANZ 1F n—1 Jump if ATOM(T) = 1. 

15 LD2 0,3(BLINK) t2 T < BLINK (Q). 

16 ST1 0,3(BLINK) t2 BLINK(Q) =P. 

17 ENT1 0,3 tə P< Q. 

18 JMP E6 t2 

19 1H STZ 0,2(ATOM) tı ATOM(T) = 0. 

20 LD2 0,3(ALINK) ty T < ALINK(Q). 

21 ST1 0,3(ALINK) ti ALINK(Q) =P. 

22 ENT1 0,3 ty P< Q. 

23 E5 LD3 0,1(BLINK) n E5. Down BLINK. Q + BLINK (P). 
24 J3Z E6 n Jump if Q = A. 

25 LDA 0,3 n — be 

26 STX 0,3(MARK) n — b2 MARK(Q) & 1. 

OT JANP E6 n— b2 Jump if NODE(Q) was already marked. 
28 LDA 0,3(ATOM) t2 + a2 

29 JANZ E6 tə +a Jump if ATOM(Q) = 1. 

30 ST2 0,1(BLINK) ta BLINK(P) =T. 

31 E4A ENT2 0,1 n—1 T +P. 

32 ENT1 0,3 n—1 P< Q. 

33 E4 LD3 0,1(ALINK) n E4. Down ALINK. Q + ALINK(P). 
34 J3Z E5 n Jump if Q = A. 

35 LDA 0,3 n — bı 

36 STX 0,3(MARK) n— bı MARK(Q) < 1. 

37 JANP E5 n—b;, Jump if NODE(Q) was already marked. 
38 LDA 0,3(ATOM) ti + aı 

39 JANZ E5 ti +aı Jump if ATOM(Q) = 1. 

40 STX 0,1(ATOM) ti ATOM(P) «+ 1. 

41 ST2 0,1(ALINK) th ALINK(P) < T. 

42 JMP E4A ti TePPeQ,toE4 J 


By Kirchhoff’s law, tı + t2 +1 = n. The total time is (34n + 4tı + 3a — 5b —8)u, where 
n is the number of nonatomic nodes marked, a is the number of atoms marked, b is 
the number of A links encountered in marked nonatomic nodes, and tı is the number 
of times we went down an ALINK (0 < ti < n). 


5. (The following is the fastest known marking algorithm for a one-level memory.) 


S1. Set MARK(PO) + 1. If ATOM(PO) = 1, the algorithm terminates; otherwise set 
s + 0, R + PO, Te A. 


S2. Set P + BLINK(R). If P = A or MARK(P) = 1, go to S3. Otherwise set 
MARK (P) + 1. Now if ATOM(P) = 1, go to S3; otherwise if S < N set S + S+ 1, 
STACK[S] <+ P, and go to 83; otherwise go to S5. 


2.3.5 ANSWERS TO EXERCISES 603 


S3. Set P «+ ALINK(R). If P = A or MARK(P) = 1, go to S4. Otherwise set 
MARK(P) «+ 1. Now if ATOM(P) = 1, go to S4; otherwise set R + P and return 
to S2. 

S4. If S = 0, terminate the algorithm; otherwise set R + STACK[S], S «+ S- 1, 
and go to 82. 

S5. Set Q + ALINK(P). If Q = A or MARK(Q) = 1, go to S6. Otherwise set 
MARK(Q) + 1. Now if ATOM(Q) = 1, go to S6; otherwise set ATOM(P) + 1, 
ALINK(P) + T, T & P, P< Q, go to S5. 

S6. Set Q + BLINK(P). If Q = A or MARK(Q) = 1, go to S7; otherwise set 
MARK(Q) «+ 1. Now if ATOM(Q) = 1, go to S7; otherwise set BLINK(P) + T, 
T + P, P + Q, go to S5. 

S7. If T = A, go to $3. Otherwise set Q + T. If ATOM(Q) = 1, set ATOM(Q) + 0, 
T «+ ALINK(Q), ALINK(Q) + P, P+ Q, and return to S6. If ATOM(Q) = 0, set 
T < BLINK(Q), BLINK(Q) + P, P + Q, and return to S7. I 

Reference: CACM 10 (1967), 501-506. 

6. From the second phase of garbage collection (or perhaps also the initial phase, 
if all mark bits are set to zero at that time). 

7. Delete steps E2 and E3, and delete “ATOM(P) + 1” in E4. Set MARK(P) + 1 in 
step E5 and use “MARK(Q) = 0”, “MARK(Q) = 1” in step E6 in place of the present 
“ATOM(Q) = 1”, “ATOM(Q) = 0” respectively. The idea is to set the MARK bit only after 
the left subtree has been marked. This algorithm works even if the tree has overlapping 
(shared) subtrees, but it does not work for all recursive List structures such as those 
with NODE(ALINK(Q)) an ancestor of NODE(Q). (Note that ALINK of a marked node is 
never changed.) 

8. Solution 1: Analogous to Algorithm E, but simpler. 

F1. Set T + A, P + PO. 

F2. Set MARK(P) + 1, and set P+ P + SIZE (P). 

F3. If MARK (P) = 1, go to F5. 

F4. Set Q + LINK(P). If Q Æ A and MARK(Q) = 0, set LINK (P) + T,T P, PQ 
and go to F2. Otherwise set P +— P — 1 and return to F3. 

F5. If T = A, stop. Otherwise set Q < T, T + LINK (Q), LINK (Q) + P,P Q-1, 
and return to F3. J 

A similar algorithm, which sometimes decreases the storage overhead and which 


avoids all pointers into the middle of nodes, has been suggested by Lars-Erik Thorelli, 
BIT 12 (1972), 555-568. 


Solution 2: Analogous to Algorithm D. For this solution, we assume that the SIZE 
field is large enough to contain a link address. Such an assumption is probably not 
justified by the statement of the problem, but it lets us use a slightly faster method 
than the first solution when it is applicable. 

G1. Set T 4 A, MARK(PO) + 1, P + PO + SIZE (P0). 

G2. If MARK(P) = 1, go to G5. 

G3. Set Q + LINK (P), P+ P — 1. 

G4. If Q # A and MARK(Q) = 0, set MARK(Q) + 1, S + SIZE(Q), SIZE(Q) < T, 

T4 Q +8. Go back to G2. 
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G5. If T = A, stop. Otherwise set P + T and find the first value of Q = P, P—1, 
P—2,... for which MARK(Q) = 1; set T< SIZE(Q) and SIZE(Q) «+ P—Q. Go 
back to G2. I 

9. H1. Set L 4+ 0, K 4+ M+ 1, MARK(O) + 1, MARK(M+ 1) + 0. 

H2. Increase L by one, and if MARK(L) = 1 repeat this step. 

H3. Decrease K by one, and if MARK(K) = 0 repeat this step. 

H4. If L > K, go to step H5; otherwise set NODE(L) 4 NODE(K), ALINK(K) < L, 
MARK (K) + 0, and return to H2. 

H5. For L = 1, 2, ..., K do the following: Set MARK(L) «+ 0. If ATOM(L) = 0 
and ALINK(L) > K, set ALINK(L) < ALINK(ALINK(L)). If ATOM(L) = 0 and 
BLINK(L) > K, set BLINK(L) «+ ALINK(BLINK(L)). I 

See also exercise 2.5-33. 

10. Z1. [Initialize.] Set F + PO, R < AVAIL, NODE(R) + NODE(F), REF(F) + R. (Here 
F and R are pointers for a queue set up in the REF fields of all header nodes 
encountered.) 

Z2. [Begin new List.] Set P+ F, Q + REF (P). 

Z3. [Advance to right.] Set P + RLINK(P). If P = A, go to Z6. 

Z4. [Copy one node.] Set Q1 < AVAIL, RLINK(Q) + Q1, Q + Q1, NODE(Q) + 
NODE(P). 

Z5. [Translate sub-List link.] If T(P) = 1, set P1 + REF (P), and if REF (P1) = A set 
REF(R) < Pi, R < AVAIL, REF(P1) + R, NODE(R) + NODE(P1), REF(Q) < R. 
If T(P) = 1 and REF(P1) # A, set REF(Q) + REF(P1). Go to Z3. 

Z6. [Move to next List.] Set RLINK(Q) + A. If REF(F) ÆR, set F + REF(REF(F) ) 
and return to Z2. Otherwise set REF (R) + A, P + PO. 

Z7. [Final cleanup.] Set Q + REF(P). If Q Æ A, set REF(P) + A and P + Q and 

repeat step Z7. J 


Of course, this use of the REF fields makes it impossible to do garbage collection with 
Algorithm D; moreover, Algorithm D is ruled out by the fact that the Lists aren’t 
well-formed during the copying. 

Several elegant List-moving and List-copying algorithms that make substantially 
weaker assumptions about List representation have been devised. See D. W. Clark, 
CACM 19 (1976), 352-354; J. M. Robson, CACM 20 (1977), 431-433. 


11. Here is a pencil-and-paper method that can be written out more formally to answer 
the problem: First attach a unique name (e.g., a capital letter) to each List in the given 
set; in the example we might have A = (a:C,b,a: F), F = (b: D), B = (a: F,b,a: E), 
C = (b: G), G = (a:C), D = (a: F), E = (b:G). Now make a list of pairs of List 
names that must be proved equal. Successively add pairs to this list until either a 
contradiction is found because we have a pair that disagree on the first level (then the 
originally given Lists are unequal), or until the list of pairs does not imply any further 
pairs (then the originally given Lists are equal). In the example, this list of pairs would 
originally contain only the given pair, AB; then it gets the further pairs CF, EF (by 
matching A and B), DG (from CF); and then we have a self-consistent set. 

To prove the validity of this method, observe that (i) if it returns the answer 
“unequal”, the given Lists are unequal; (ii) if the given Lists are unequal, it returns 
the answer “unequal”; (iii) it always terminates. 
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12. When the AVAIL list contains N nodes, where N is a specified constant to be 
chosen as discussed below, initiate another coroutine that shares computer time with 
the main routine and does the following: (a) Marks all N nodes on the AVAIL list; (b) 
marks all other nodes that are accessible to the program; (c) links all unmarked nodes 
together to prepare a new AVAIL list for use when the current AVAIL list is empty, and 
(d) resets the mark bits in all nodes. One must choose N and the ratio of time sharing 
so that operations (a), (b), (c), and (d) are guaranteed to be complete before N nodes 
are taken from the AVAIL list, yet the main routine is running sufficiently fast. It is 
necessary to use some care in step (b) to make sure that all nodes “accessible to the 
program” are included, as the program continues to run; details are omitted here. If 
the list formed in (c) has fewer than N nodes, it may be necessary to stop eventually 
because memory space might become exhausted. [For further information, see Guy L. 
Steele Jr., CACM 18 (1975), 495-508; P. Wadler, CACM 19 (1976), 491-500; E. W. 
Dijkstra, L. Lamport, A. J. Martin, C. S. Scholten, and E. F. M. Steffens, CACM 21 
(1978), 966-975; H. G. Baker, Jr., CACM 21 (1978), 280-294.] 


SECTION 2.4 
1. Preorder. 
2. It is essentially proportional to the number of Data Table entries created. 
3. Change step A5 to: 


A5’. [Remove top level.] Remove the top stack entry; and if the new level number 
at the top of the stack is > L, let (L1,P1) be the new entry at the top of the 
stack and repeat this step. Otherwise set SIB(P1) + Q and then let (L1, P1) 
be the new entry at the top of the stack. 


4. (Solution by David S. Wise.) Rule (c) is violated if and only if there is a data item 
whose complete qualification Ao OF ... OF An is also a COBOL reference to some other 
data item. Since the parent A; OF... OF An must also satisfy rule (c), we may assume 
that this other data item is a descendant of the same parent. Therefore Algorithm A 
would be extended to check, as each new data item is added to the Data Table, whether 
its parent is an ancestor of any other item of the same name, or if the parent of any 
other item of the same name is in the stack. (When the parent is A, it is everybody’s 
ancestor and always on the stack.) 

On the other hand, if we leave Algorithm A as it stands, the COBOL programmer 
will get an error message from Algorithm B when trying to use an illegal item. Only 
MOVE CORRESPONDING can make use of such items without error. 


5. Make these changes: Step replace by 
B1. P 4 LINK( Po) P 4+ LINK(INFO(T)) 
B2. k0 KT 
B3. k<n RLINK(K) # A 
B4. k+k+1 K + RLINK (K) 
B6. NAME(S) = Pk NAME(S) = INFO(K) 


6. A simple modification of Algorithm B makes it search only for complete references 
(if k = n and PARENT (S) 4 A in step B3, or if NAME(S) 4 Px in step B6, set P + PREV (P) 
and go to B2). The idea is to run through this modified Algorithm B first; then, if Q is 
still A, to perform the unmodified algorithm. 


7. MOVE MONTH OF DATE OF SALES TO MONTH OF DATE OF PURCHASES. MOVE DAY OF DATE 
OF SALES TO DAY OF DATE OF PURCHASES. MOVE YEAR OF DATE OF SALES TO YEAR OF 
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DATE OF PURCHASES. MOVE ITEM OF TRANSACTION OF SALES TO ITEM OF TRANSACTION OF 
PURCHASES. MOVE QUANTITY OF TRANSACTION OF SALES TO QUANTITY OF TRANSACTION 
OF PURCHASES. MOVE PRICE OF TRANSACTION OF SALES TO PRICE OF TRANSACTION OF 
PURCHASES. MOVE TAX OF TRANSACTION OF SALES TO TAX OF TRANSACTION OF PURCHASES. 


8. If and only if a or ĝ is an elementary item. (It may be of interest to note that 
the author failed to handle this case properly in his first draft of Algorithm C, and it 
actually made the algorithm more complicated.) 


9. “MOVE CORRESPONDING a TO 8”, if neither a nor 8 is elementary, is equivalent to 
the set of statements “MOVE CORRESPONDING A OF a TO A OF 8” taken over all names 
A common to groups a and 8. (This is a more elegant way to state the definition 
than the more traditional and more cumbersome definition of “MOVE CORRESPONDING” 
given in the text.) We may verify that Algorithm C satisfies this definition, using an 
inductive proof that steps C2 through C5 will ultimately terminate with P = PO and 
Q = Q0. Further details of the proof are filled in as we have done many times before in 
a “tree induction” (see, for example, the proof of Algorithm 2.3.1T). 


10. (a) Set S1 < LINK(P,). Then repeatedly set S1 4 PREV(S1) zero or more times 
until either S1 = A (NAME(S) # P,) or S1 = S (NAME(S) = P). (b) Set P1 + P and 
then set P1 <- PREV(P1) zero or more times until PREV (P1) = A; do a similar operation 
with variables Q1 and Q; then test if P1 = Q1. Alternatively, if the Data Table entries 
are ordered so that PREV(P) < P for all P, a faster test can be made in an obvious way 
depending on whether P > Q or not, following the PREV links of the larger to see if the 
smaller is encountered. 


11. A minuscule improvement in the speed of step C4 would be achieved by adding a 
new link field SIB1(P) = CHILD(PARENT(P)). More significantly, we could modify the 
CHILD and SIB links so that NAME(SIB(P)) > NAME(P); this would speed up the search 
in step C3 considerably because it would require only one pass over each family to find 
the matching members. This change would therefore remove the only “search” present 
in Algorithm C. Algorithms A and C are readily modified for this interpretation, and 
the reader may find it an interesting exercise. (However, if we consider the relative 
frequency of MOVE CORRESPONDING statements and the usual size of family groups, the 
resulting speedup will not be terribly significant in the translation of actual COBOL 
programs.) 


12. Leave steps B1, B2, B3 unchanged; change the other steps thus: 
B4’. Set k + k+1,R + LINK (P). 


B5’. If R = A, set P + PREV(P) and go to B2’ (we haven’t found a match). If 
R < S < SCOPE(R), set S + R and go to B3’. Otherwise set R + PREV (R) and 
repeat step B5’. I 
This algorithm does not adapt to the PL/I convention of exercise 6. 


13. Use the same algorithm, minus the operations that set NAME, PARENT, CHILD, 
and SIB. Whenever removing the top stack entry in step A5, set SCOPE (P1) + Q— 1. 
When the input is exhausted in step A2, simply set L + 0 and continue, then terminate 
the algorithm if L = 0 in step A7. 


14. The following algorithm, using an auxiliary stack, has steps numbered to show a 
direct correspondence with the text’s algorithm. 


C1’. Set P + PO, Q + QO, and set the stack contents empty. 
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C2’. If SCOPE(P) = P or SCOPE(Q) = Q, output (P,Q) as one of the desired pairs 
and go to C5’. Otherwise put (P,Q) on the stack and set P+ P+1,Q¢ Q+1. 


C3’. Determine if P and Q point to entries with the same name (see exercise 10(b)). 
If so, go to C2’. If not, let (P1,Q1) be the entry at the top of the stack; if 
SCOPE(Q) < SCOPE(Q1), set Q + SCOPE(Q) + 1 and repeat step C3’. 


C4’. Let (P1,Q1) be the entry at the top of the stack. If SCOPE(P) < SCOPE(P1), set 
P + SCOPE(P) +1, Q + Q1+ 1, and go back to C3’. If SCOPE(P) = SCOPE(P1), 
set P+ P1, Q + Q1 and remove the top entry of the stack. 


C5’. If the stack is empty, the algorithm terminates. Otherwise go to C4’. J 


SECTION 2.5 


1. In such fortuitous circumstances, a stack-like operation may be used as follows: 
Let the memory pool area be locations 0 through M — 1, and let AVAIL point to the 
lowest free location. To reserve N words, report failure if AVAIL + N > M, otherwise set 
AVAIL < AVAIL +N. To free these N words, just set AVAIL < AVAIL — N. 

Similarly, cyclic queue-like operation is appropriate for a first-in-first-out discipline. 


2. The amount of storage space for an item of length | is k[1/(k — b)], which has the 
average value kL /(k — b) + (1 — a)k, where a is assumed to be 1/2, independent of k. 
This expression is a minimum (for real values of k) when k = b+ V2bL. So choose k to 
be the integer just above or just below this value, whichever gives the lowest value of 
kL/(k — b)+ 4k. For example, if b = 1 and L = 10, we would choose k ~ 1+ v20 = 5 
or 6; both are equally good. For much greater detail about this problem, see JACM 
12 (1965), 53-70. 


4. rll =Q, rl2=P. 


A1 LDA N rA &N. 
ENT2 AVAIL P 4+ LOC (AVAIL). 
A2A ENT1 0,2 Q<P. 


A2 LD2 0,1(LINK) P< LINK(Q). 
J2N OVERFLOW If P= A, no room. 
A3 CMPA 0,2(SIZE) 


JG A2A Jump if N > SIZE(P). 
A4 SUB 0,2(SIZE) rA + N-— SIZE(P) =K. 
JANZ *+3 Jump if K # 0. 


LDX 0,2(LINK) 

STX 0,1(LINK) LINK(Q) + LINK(P). 
STA 0,2(SIZE) SIZE(P) +K. 

LD1 0,2(SIZE) Optional ending, 

INC1 0,2 sets ril + P +K. I 


5. Probably not. The unavailable storage area just before location P will subsequently 
become available, and its length will be increased by the amount K; an increase of 99 
would not be negligible. 


6. The idea is to try to search in different parts of the AVAIL list each time. We can 
use a “roving pointer,” called ROVER for example, which is treated as follows: In step 
Al, set Q 4+ ROVER. After step A4, set ROVER + LINK(Q) if LINK(Q) Æ A, otherwise 
set ROVER < LOC(AVAIL). In step A2, when P = A the first time during a particular 
execution of Algorithm A, set Q + LOC(AVAIL) and repeat step A2. When P = A the 
second time, the algorithm terminates unsuccessfully. In this way ROVER will tend to 
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point to a random spot in the AVAIL list, and the sizes will be more balanced. At the 
beginning of the program, set ROVER +— LOC (AVAIL); it is also necessary to set ROVER to 
LOC(AVAIL) everywhere else in the program where the block whose address equals the 
current setting of ROVER is taken out of the AVAIL list. (Sometimes, however, it is useful 
to have small blocks at the beginning, as in the strict first-fit method; for example, we 
might want to keep a sequential stack at the high end of memory. In such cases we can 
reduce the search time by using trees as suggested in exercise 6.2.3-30.) 


7. 2000, 1000 with requests of sizes 800, 1300. [An example where worst-fit succeeds, 
while best-fit fails, has been constructed by R. J. Weiland.] 


8. In step Al”, also set M + œo, R + A. In step A2”, if P = A go to A6”. In step 
A3”, go to A5” instead of to A4”. Add new steps as follows: 


A5”. [Better fit?] If M > SIZE(P), set R + Q and M + SIZE(P). Then set Q + P 
and return to A2”. 


A6”. [Any found?] If R = A, the algorithm terminates unsuccessfully. Otherwise 
set Q + R, P+ LINK(Q), and goto A4”. J 


9. Obviously if we are so lucky as to find SIZE(P) = N, we have a best fit and it is not 
necessary to search farther. (When there are only very few different block sizes, this 
occurs rather often.) If a “boundary tag” method like Algorithm C is being used, it is 
possible to maintain the AVAIL list in sorted order by size; so the length of search could 
be cut down to half the length of the list or less, on the average. But the best solution 
is to make the AVAIL list into a balanced tree structure as described in Section 6.2.3, if 
it is expected to be long. 


10. Make the following changes: 

Step B2, for “P > PO” read “P > PO”. 

At the beginning of step B3, insert “If PO + N > P and P Æ A, set N + max(N, 
P+SIZE(P) — PO), P+ LINK(P), and repeat step B3.” 

Step B4, for “Q + SIZE(Q) = PO”, read “Q + SIZE(Q) > PO”; and for “SIZE(Q) + 
SIZE(Q) +N” read “SIZE(Q) + max(SIZE(Q) ,;PO+N— Q)”. 


11. If PO is greater than ROVER, we can set Q + ROVER instead of Q < LOC(AVAIL) in 
step B1. If there are n entries in the AVAIL list, the average number of iterations of 
step B2 is (2n + 3)(n + 2)/(6n+6) = 3n+2+O(+). For example if n = 2 we get 9 
equally probable situations, where P1 and P2 point to the two existing available blocks: 


PO < P1 P1 < PO < P2 P2 < PO 


ROVER = P1 1 | 1 2 
ROVER = P2 1 2 1 


ROVER = LOC (AVAIL) 1 2 3 


This chart shows the number of iterations needed in each case. The average is 


5 (0) +G)+G)+G)+G))=5(G)+G)) <5 


A2*. If P = LOC(AVAIL) and F = 0, set P + AVAIL, F + 1, and repeat step A2*. 
If P = LOC(AVAIL) and F Æ 0, the algorithm terminates unsuccessfully. 


A8*. If SIZE(P) > N, go to A4*; otherwise set P + LINK(P) and return to A2*. 
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A4*. Set ROVER + LINK (P), K «+ SIZE(P) —N. IfK < c (where c is a constant > 2), 
set LINK(LINK(P + 1)) < ROVER, LINK(ROVER + 1) + LINK(P +1), L + P; 
otherwise set L + P + K, SIZE(P) «+ SIZE(L— 1) + K, TAG(L—1) «+ “—”, 
SIZE(L) + N. Finally set TAG(L) + TAG(L+ SIZE(L) —1) «+ “+”. J 


13. rfl 


P, rX 


F, rI2 =L. 


LINK 
SIZE 
TSIZE 
TAG 
A1 


A3 


A2 


A4 


1H 


2H 


EQU 
EQU 
EQU 
EQU 
LDA 
SLA 
ENTX 
LD1 
JMP 
CMPA 
JLE 
LD1 
ENT2 
J2NZ 
JXNZ 
ENTX 
LD1 
JMP 
LD2 
ST2 
LDA 
SUB 
CMPA 
JGE 
LD3 
ST2 
ST3 
ENT2 
LD3 
JMP 
STA 
LD2 
INC2 
LDAN 
STA 
LD3 
ST3 
INC3 
STZ 


:5 


2 
:2 
(0) 


owzoo0or œ 


ROVER 
A2 

0,1 (SIZE) 
A4 
0,1(LINK) 
-AVAIL,1 
A3 
OVERFLOW 
1 
AVAIL(LINK) 
A2 

0,1 (LINK) 
ROVER 

0,1 (SIZE) 
N 

ae 

1F 
1,1(LINK) 
0,3(LINK) 
1,2(LINK) 
0,1 
0,1(SIZE) 
2F 
0,1(SIZE) 
0,1(SIZE) 
0,1 
0,1(SIZE) 
-1,2(TSIZE) 
N 
0,2(TSIZE) 
0,2 
-1,3(TAG) 


rÀ +N. 

Shift into SIZE field. 
F +0. 

P + ROVER. 


Jump if N < SIZE(P). 
P< LINK(P). 
rl2 + P — LOC(AVAIL). 


Is F 40? 
Set F< 1. 
P + AVAIL. 


ROVER <+ LINK(P). 
rA =K + SIZE(P) —N. 


Jump if K > c. 

rI3 + LINK(P + 1). 
LINK (r13) < ROVER. 
LINK (ROVER + 1) + rI3. 
L <P. 

rl3 + SIZE(P). 


SIZE(P) +K. 


LPK: 

rA + —K. 

SIZE(L — 1) + K, TAG(L—1) = “—”. 
rI3 + N. 

TAG(L) + “+”, also set SIZE(L) < rI3. 


TAG(L+SIZE(L) -—1) + “+”. J 


14. (a) This field is needed to locate the beginning of the block, in step C2. It could 
be replaced (perhaps to advantage) by a link to the first word of the block. See also 
exercise 19. (b) This field is needed because we sometimes need to reserve more than N 
words (for example if K = 1), and the amount reserved must be known when the block 
is subsequently freed. 
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15, 16. 


Ci 


C2 


c3 


c4 


c5 


c6 


C7 


c8 
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rll = PO, rl2 = P1, rl3 = F, rl4 = B, rI6 N. 

LD1 PO Cl. 

LD2 0,1(SIZE) 

ENN6 0,2 N «+ SIZE(PO). 

INC2 0,1 P1 4 PO +N. 

LD5 0,2(TSIZE) 

J5N C4 To C4 if TAG(P1) = “—”. 

LD5 -1,1(TSIZE) C2. 

J5N C7 To C7 if TAG(PO — 1) = “—”. 
LD3 AVAIL(LINK) C3. Set F < AVAIL. 

ENT4 AVAIL B «+ LOC(AVAIL). 

JMP C5 To Cbd. 

INC6 0,5 C4. Ne N+SIZE(P1). 

LD3 0,2(LINK) F + LINK(P1). 

LD4 1,2(LINK) B 4+ LINK(P1 +1). 

CMP2 ROVER (New code, because of the ROVER 
JNE *+3 feature of exercise 12: 

ENTX AVAIL If P1 = ROVER, 

STX ROVER set ROVER + LOC (AVAIL) .) 
DEC2 0,5 P1< P1+SIZE(P1). 

LD5 -1,1(TSIZE) 

J5N C6 To C6 if TAG(PO — 1) = “—”. 
ST3 0,1(LINK) C5. LINK(PO) +F. 

ST4 1,1(LINK) LINK(PO+ 1) < B. 

ST1 1,3(LINK) LINK(F + 1) + PO. 

ST1 0,4(LINK) LINK(B) < PO. 

JMP C8 To C8. 

ST3 0,4(LINK) C6. LINK(B) +F. 

ST4 1,3(LINK) LINK(F + 1) +B. 

INC6 0,5 C7. N 4+ N + SIZE(PO — 1). 

INC1 0,5 PO + PO — SIZE (PO — 1). 

ST6 0,1(TSIZE) C8. SIZE(PO) + N, TAG(PO) + “—”. 
ST6 -1,2(TSIZE) SIZE(P1—1)<N, TAG(P1— 1) + “—”. J 


17. Both LINK fields equal to LOC (AVAIL). 


18. Algorithm A reserves the upper end of a large block. When storage is completely 
available, the first-fit method actually begins by reserving the high-order locations, but 
once these become available again they are not re-reserved since a fit is usually found 
already in the lower locations; thus the initial large block at the lower end of memory 
quickly disappears with first-fit. A large block rarely is the best fit, however, so the 
best-fit method leaves a large block at the beginning of memory. 


19. Use the algorithm of exercise 12, except delete the references to SIZE(L — 1), 
TAG(L—1), and TAG(L+SIZE(L) —1) from step A4*; also insert the following new step 


between steps A2* and A3*: 


A2.5*. Set P1 + P+SIZE(P). If TAG(P1) = “+”, proceed to step A3. Otherwise 
set P2 «+ LINK(P1), LINK(P2+1) «+ LINK(P1+1), LINK(LINK(P1+1)) <— 


P2, SIZE(P) « SIZE(P) + SIZE(P1). 


If ROVER = P1, set ROVER + P2. 


Repeat step A2.5*. 
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Clearly the situation of (2), (3), (4) can’t occur here; the only real effect on 
storage allocation is that the search here will tend to be longer than in exercise 12, 
and sometimes K will be less than c although there is really another available block 
preceding this one that we do not know about. 

(An alternative is to take the collapsing out of the inner loop A3*, and to do the 
collapsing only in step A4* before the final allocation or in the inner loop when the 
algorithm would otherwise have terminated unsuccessfully. This alternative requires a 
simulation study to see if it is an improvement or not.) 

[This method, with a few refinements, has proved to be quite satisfactory in the 
implementations of TEX and METAFONT. See TX: The Program (Addison-Wesley, 
1986), §125.] 


20. When a buddy is found to be available, during the collapsing loop, we want to 
remove that block from its AVAIL[k] list, but we do not know which links to update 
unless (i) we do a possibly long search, or (ii) the list is doubly linked. 

21. If n = 2¥a, where 1 < a < 2, an is 27** (a — 2) + §, and bp is g7h lat P 
The ratio an/bn for large n is essentially 4(a — 2)/ a’, which takes its minimum value 
$ when a = 1 and 2, and its maximum value 3 when a = 1. So an /bn approaches no 
limit; it oscillates between these two extremes. The averaging methods of Section 4.2.4 


do, however, yield an average ratio of 4(In 2)! f (a — 2) da/o® = (In2)~! ~ 1.44. 


22. This idea requires a TAG field in several words of the 11-word block, not only in 
the first word. It is a workable idea, if those extra TAG bits can be spared, and it would 
appear to be especially suitable for use in computer hardware. 


23. 011011110100; 011011100000. 


24. This would introduce a bug in the program; we may get to step S1 when TAG(0) = 1, 
since $2 may return to S1. To make it work, add “TAG(L) + 0” after “L + P” in step 82. 
(It is easier to assume instead that TAG(2”) = 0.) 


25. The idea is absolutely correct. (Criticism need not be negative.) The list heads 
AVAIL[k] may be eliminated for n < k < m; the algorithms of the text may be used if 


m” is changed to “n” in steps R1, S1. The initial conditions (13) and (14) should be 
changed to indicate 2”~” blocks of size 2” instead of one block of size 2”. 


26. Using the binary representation of M, we can easily modify the initial conditions 
(13), (14) so that all memory locations are divided into blocks whose size is a power of 
two, with blocks in decreasing order of size. In Algorithm S, TAG(P) should be regarded 
as 0 whenever P > M — 2". 

27. rll = k, rl2 = j, rI3 = j —k, rl4 = L, LOC(AVAIL[j]) = AVAIL + j; as- 
sume that there is an auxiliary table TWO[j] = 2’, stored in location TWO + j, for 
0< j <m. Assume further that “+” and “—” represent tags of 0 and 1, and that 
TAG(LOC(AVAIL[j])) = “—”; but TAG(LOC(AVAIL[m + 1])) = “+” is a sentinel. 


00 KVAL EQU 5:5 
01 TAG EQU O 
02 LINKF EQU 1 
03 LINKB EQU 3 
04 TLNKF EQU 0: 
K 
0 
0 


DO BNO 


05 R1 LD1 

06 ENT2 
07 ENT3 
08 LD4 AVAIL,2(LINKF) 


R1. Find block. 


mal jek. 
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R2 


R3 
R4 
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DONE 


. ril 


ENTS 
DECS 
J5NZ 
INC2 
INC3 
LD4N 
J4NN 
JMP 
LD5 
ST5 
ENTA 
STA 
STZ 
J3Z 
DEC3 
DEC2 
LD5 
INC5 
ENNA 
STA 
STA 
ST5 
STS 
ST2 
J3P 


AVAIL,2 
0,4 

R2 

1 

1 
AVAIL,2(TLNKF) 
1B 

OVERFLOW 
0,4(LINKB) 
AVAIL,2(LINKB) 
AVAIL,2 
0,5(LINKF) 
0,4(TAG) 

DONE 

1 

1 

TWO,2 

0,4 

AVAIL ,2 
0,5(TLNKF) 
0,5(LINKB) 
AVAIL, 2 (LINKF) 
AVAIL, 2 (LINKB) 
0,5(KVAL) 

R4 


k, r15 


P, rl4 


S1 LD4 L 


1H 


S2 


S3 


LD1 
ENT 
XOR 
STA 
LD5 
LDA 
JAN 
CMP 
JNE 
LD2 
LD3 
ST3 
ST2 
INC 
CMP 
JL 

ENT 
JMP 
LD2 
ENN. 
STA 


K 

A 0,4 
TWO 
TEM. 
TEM. 
0,5 
$3 
0,5 
$3 
0,5 
0,5 
0,2 
0,3 
1 

TEM. 
1B 
0,5 
1B 
AVA 
AVA 
0,4 


N 
1 


1 
4 


4 


A 


al 
P 
P 


(KVAL) 


(LINKB) 
(LINKF) 
(LINKF) 
(LINKB) 


P 


IL, 1(LINKF) 
IL,1 
(0:4) 


PRR RRR 
fe, ae. es iS. Ae oh 
fe a Si a a E. 
RNRNNANV 


w w 
UU 


14 
14 


T T 


en 
li 
T 


DIII 


DPHDDDVDVDVIVAIADPPPHH HS 


— 


FPP RRHNKHKHHAKHAHYV 


2.5 


DII 


Jump if AVAILF[j] Æ LOC(AVAIL[Jj]). 
Increase j. 
Is j < m? 


R2. Remove from list. 
AVAILB[7] < LINKB(L). 


LINKF(L) + LOC(AVAIL[j]). 
TAG(L) + 0. 

R3. Split required? 

R4. Split. 

Decrease j. 

rl5 =P. 

PL+. 


TAG(P) + 1, LINKF (P) + LOC(AVAIL[j]). 
LINKB(P) «+ LOC(AVAIL[j]). 
AVAILF[j] +P. 
AVAILB[j] +P. 
KVAL(P) < j. 
Go to R3. 
I 


L; assume TAG(2™) = “+”. 


S1. Is buddy available? 


rA + buddy, (L). 
P<rA. 
Jump if TAG(P) = 0. 


Jump if KVAL(P) Æ k. 
S2. Combine with buddy. 


LINKF (LINKB(P)) + LINKF (P). 
LINKB(LINKF(P)) + LINKB (P). 
Increase k. 

If L >P, set Le P. 


S53. Put on list. 


TAG(L) «+ 1, LINKB(L) + LOC(AVAIL[K]). 
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23 ST2 0,4(LINKF) 1 LINKF (L) «+ AVAILF [k]. 
24 ST1 0,4(KVAL) 1 KVAL (L) & k. 

25 ST4 0,2(LINKB) 1 LINKB(AVAILF[k]) < L. 
26 ST4 AVAIL,1(LINKF) 1 AVAIL[k] —L. I 


29. Yes, but only at the expense of some searching, or (better) an additional table 
of TAG bits packed somehow. (It is tempting to suggest that buddies not be joined 
together during Algorithm S, but only in Algorithm R if there is no block large enough 
to meet the request; but that would probably lead to a badly fragmented memory.) 


31. See David L. Russell, SICOMP 6 (1977), 607-621. 
32. Steven Crain points out that the method always frees all blocks and starts afresh 


before 16667 units of time have elapsed; hence the stated limit certainly exists. Proof: 
Let Un = n+ tn, so that gn = |$ min(10000, f (un-1 — n), f(un-2—7),---, f(wo—n))]. 


Let zo = 0 and xı = uo, and £k+ı = Max(uo, ..., Usp—1) for k > 1. If £k > £k—1 then 
1 1 
Un <n 5 Fær n) = ak ars th g7! for £k—1 <n < Tk; 


therefore £k+1 — £k < + (2k —2Xk-1), and we must have £k = tp-1 before reaching time 

12500 + [12500/4] + [12500/47| +---. 

33. G1. [Clear LINKs.] Set P ¢ 1, and repeat the operation LINK(P) + A, P + 
P + SIZE(P) until P = AVAIL. (This merely sets the LINK field in the first 
word of each node to A; we may assume in most cases that this step is 
unnecessary, since LINK(P) is set to A in step G9 below and it can be set to 
A by the storage allocator.) 


G2. [Initialize marking phase.] Set TOP + USE, LINK(TOP) «+ AVAIL, LINK (AVAIL) 
+ A. (TOP points to the top of a stack as in Algorithm 2.3.5D.) 


G3. [Pop up stack.] Set P + TOP, TOP + LINK(TOP). If TOP = A, go to G5. 


G4. [Put new links on stack.| For 1 < k < T(P), do the following operations: 
Set Q < LINK(P + k); then if Q # A and LINK(Q) = A, set LINK(Q) + TOP, 
TOP + Q. Then go back to G3. 


G5. [Initialize next phase.] (Now P = AVAIL, and the marking phase has been 
completed so that the first word of each accessible node has a nonnull LINK. 
Our next goal is to combine adjacent inaccessible nodes, for speed in later 
steps, and to assign new addresses to the accessible nodes.) Set Q + 1, 
LINK(AVAIL) + Q, SIZE(AVAIL) + 0, P + 1. (Location AVAIL is being used 
as a sentinel to signify the end of a loop in subsequent phases.) 


G6. [Assign new addresses.] If LINK(P) = A, go to G7. Otherwise if SIZE(P) = 0, 
go to G8. Otherwise set LINK(P) + Q, Q + Q+SIZE(P), P + P + SIZE(P), 
and repeat this step. 


G7. [Collapse available areas.] If LINK(P + SIZE(P)) = A, increase SIZE(P) by 
SIZE(P + SIZE(P)) and repeat this step. Otherwise set P + P+SIZE(P) and 
return to G6. 


G8. [Translate all links.] (Now the LINK field in the first word of each accessible 
node contains the address to which the node will be moved.) Set USE + 
LINK(USE), and AVAIL «+ Q. Then set P + 1, and repeat the following 
operation until SIZE(P) = 0: If LINK(P) # A, set LINK(Q) < LINK (LINK(Q)) 
for all Q such that P < Q<P+T(P) and LINK(Q) Æ A; then regardless of the 
value of LINK(P), set P+ P+ SIZE(P). 
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G9. [Move.] Set P + 1, and repeat the following operation until SIZE(P) = 0: 
Set Q + LINK(P), and if Q Æ A set LINK(P) + A and NODE(Q) + NODE(P); 
then whether Q = A or not, set P+ P+ SIZE(P). (The operation NODE(Q) + 
NODE(P) implies the movement of SIZE(P) words; we always have Q < P, so it 
is safe to move the words in order from smallest location to largest.) J 


[This method is called the “LISP 2 garbage collector.” An interesting alternative, which 
does not require the LINK field at the beginning of a node, can be based on the idea 
of linking together all pointers that point to each node—see Lars-Erik Thorelli, BIT 
16 (1976), 426-441; R. B. K. Dewar and A. P. McCann, Software Practice & Exp. 
7 (1977), 95-113; F. Lockwood Morris, CACM 21 (1978), 662-665, 22 (1979), 571; 
H. B. M. Jonkers, Inf. Proc. Letters 9 (1979), 26-30; J. J. Martin, CACM 25 (1982), 
571-581; F. Lockwood Morris, Inf. Proc. Letters 15 (1982), 139-142, 16 (1983), 215. 
Other methods have been published by B. K. Haddon and W. M. Waite, Comp. J. 10 
(1967), 162-165; B. Wegbreit, Comp. J. 15 (1972), 204-208; D. A. Zave, Inf. Proc. 
Letters 3 (1975), 167-169. Cohen and Nicolau have analyzed four of these approaches 
in ACM Trans. Prog. Languages and Systems 5 (1983), 532-553.] 


34. Let TOP = rll, Q = rl2, P = rI3, k = rI4, SIZE(P) = rI5. Assume further that 
A= 0, and LINK(0) Æ 0 to simplify step G4. Step G1 is omitted. 


01 LINK EQU 4:5 
02 INFO EQU 0:3 
03 SIZE EQU 1:2 
04 T EQ 3:3 


05 G2 LD1 USE 1 G2. Initialize marking phase. TOP < USE. 
06 LD2 AVAIL 1 

O7 ST2 0,1(LINK) 1 LINK(TOP) + AVAIL. 

08 STZ 0,2(LINK) 1 LINK(AVAIL) + A. 

09 G3 ENT3 0,1 a+1 G3. Pop up stack. P < TOP. 

10 LD1 0,1(LINK) a+1 TOP + LINK(TOP). 

11 J1iZ G5 a+1 To G5 if TOP = A. 

12 G4 LD4 0,3(T) a G4. Put new links on stack. k < T(P). 
13 1H J4Z G3 a+b k=0? 

14 INC3 1 b P&P+1. 

15 DEC4 1 b kek-l. 

16 LD2 0,3(LINK) b Q + LINK(P). 

17 LDA 0,2(LINK) b 

18 JANZ 1B b Jump if LINK(Q) # A. 

19 ST1 0,2(LINK) a—1 Otherwise set LINK(Q) < TOP, 

20 ENT1 0,2 a—1 TOP & Q. 

21 JMP 1B a—1 

22 GS ENT2 1 1 G5. Initialize next phase. Q + 1. 

28 ST2 0,3 1 LINK(AVAIL) + 1, SIZE(AVAIL) + 0. 
24 ENT3 1 1 Pel. 

25 JMP G6 1 

26 1H ST2 0,3(LINK) a LINK(P) + Q. 

27 INC2 0,5 a Q + Q + SIZE (P). 

28 INC3 0,5 a P + P + SIZE(P). 


29 G6 LDA 0,3(LINK) a+1 G6. Assign new addresses. 
30 G6A LD5 0,3(SIZE) a+c+1 
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31 JAZ G7 a+c+1 Jump if LINK(P) = A. 

32 J5NZ 1B a+1 Jump if SIZE(P) Æ 0. 

83 G8 LD1 USE 1 G8. Translate all links. 

34 LDA 0,1(LINK) 1 

35 STA USE 1 USE + LINK(USE). 

36 ST2 AVAIL 1 AVAIL & Q. 

37 ENT3 1 1 Pel. 

38 JMP G8P 1 

39 1H LD6 0,6(SIZE) d 

40 INC5 0,6 d rI5 + rI5 + SIZE(P + SIZE(P)). 
41 G7 ENT6 0,3 c+d G7. Collapse available areas. 
42 INC6 0,5 c+d rl6 + P+ SIZE(P). 

43 LDA 0,6(LINK) c+d 

44 JAZ 1B c+d Jump if LINK(rI6) = A. 

45 ST5 0,3(SIZE) c SIZE(P) + rI5. 

46 INC3 0,5 c P + P + SIZE(P). 

47 JMP G6A c 

48 2H DEC4 1 b kek-l. 

49 INC2 1 b Qg+l. 

50 LD6 0,2(LINK) b 

51 LDA 0,6(LINK) b 

52 STA 0,2(LINK) b LINK(Q) «+ LINK(LINK(Q)). 
538 1H J4NZ 2B a+b Jump if k £0. 

54 3H INC3 0,5 a+c P + P + SIZE(P). 

55 G8P LDA 0,3(LINK) 1+a+c 

56 LD5 0,3(SIZE) lt+a+c 

57 JAZ 3B 1l+a+c Is LINK(P) = A? 

58 LD4 0,3(T) 1l+a k + T(P). 

59 ENT2 0,3 l+a Q<P. 

60 J5NZ 1B l+a Jump unless SIZE(P) = 0. 
61 G9 ENT3 1 1 G9. Move. P + 1. 

62 ENT1 1 1 Set rll for MOVE instructions. 
63 JMP G9P 1 

64 1H STZ 0,3(LINK) a LINK(P) + A. 

65 ST5 *+1(4:4) a 

66 MOVE 0,3(*) a NODE(rI1) + NODE(P), rll + rll +SIZE(P). 
67 3H INC3 0,5 a+c P< P+SIZE(P). 

68 G9P LDA 0O,3(LINK) 1l+a+c 

69 LD5 0,3(SIZE) lt+a+c 

70 JAZ 3B 1+a+c Jump if LINK(P) = A. 

71 J5NZ 1B l+a Jump unless SIZE(P) = 0. J 


In line 66 we are assuming that the size of each node is sufficiently small that it can 
be moved with a single MOVE instruction; this seems a fair assumption for most cases 
when this kind of garbage collection is applicable. 

The total running time for this program is (44a+17b+2w-+ 25c+ 8d+ 47) u, where 
a is the number of accessible nodes, b is the number of link fields therein, c is the number 
of inaccessible nodes that are not preceded by an inaccessible node, d is the number of 
inaccessible nodes that are preceded by an inaccessible node, and w is the total number 
of words in the accessible nodes. If the memory contains n nodes, with pn of them 


616 ANSWERS TO EXERCISES 2.5 


inaccessible, then we may estimate a = (1 — p)n, c = (1 — p)pn, d = p?n. Example: 
five-word nodes (on the average), with two link fields per node (on the average), and a 
memory of 1000 nodes. Then when p = 0.2, it takes 374u per available node recovered; 
when p = 0.5, it takes 104u; and when p = 0.8, it takes only 33u. 


36. A single customer will be able to sit in one of the sixteen seats 1, 3, 4, 6, ..., 23. 
If a pair enters, there must be room for them; otherwise there are at least two people 
in seats (1,2,3), at least two in (4,5,6), ..., at least two in (19, 20,21), and at least 
one in 22 or 23, so at least fifteen people are already seated. 


37. First sixteen single males enter, and she seats them. There are 17 gaps of empty 
seats between the occupied seats, counting one gap at each end, with a gap of length 
zero assumed between adjacent occupied seats. The total number of empty seats, 
namely the sum of all seventeen gaps, is 6. Suppose x of the gaps are of odd length; 
then 6 — x spaces are available to seat pairs. (Note that 6 — x is even and > 0.) Now 
each of the customers 1, 3, 5, 7, 9, 11, 13, 15, from left to right, who has an even gap 
on both sides, finishes his lunch and walks out. Each odd gap prevents at most one of 
these eight diners from leaving, hence at least 8 — x people leave. There still are only 
6 — x spaces available to seat pairs. But now (8 — x)/2 pairs enter. 


38. The arguments generalize readily; N(n,2) = |(3n — 1)/2] for n > 1. [When the 
hostess uses a first-fit strategy instead of an optimal one, Robson has proved that the 
necessary and sufficient number of seats is |(5n — 2)/3].] 


39. Divide memory into three independent regions of sizes N(n1,m), N(n2, m), and 
N(2m — 2, m). To process a request for space, put each block into the first region for 
which the stated capacity is not exceeded, using the relevant optimum strategy for that 
region. This cannot fail, for if we were unable to fill a request for x locations we must 
have at least (nı — £ +1) + (na — x +1) + (2m — z — 1) > nı + nə — z locations already 
occupied. 

Now if f(n) = N(n,m)+N(2m—2, m), we have the subadditive law f (nı +n2) < 
f(m1) + f(n2). Hence lim f(n)/n exists. (Proof: f(a + bc) < f(a) + bf(c); hence 
lim SUPp oo f(n)/n = maxo<a<c limsup,_,,, f(a+bc)/(a+bc) < f(c)/c for all c; hence 
lim sup,,_,., f(n)/n < lim infra f(n)/n.) Therefore lim N(n,m)/n exists. 

[From exercise 38 we know that N(2) = 3. The value N(m) is not known for any 
m > 2. It is not difficult to show that the multiplicative factor for just two block sizes, 
1 and b, is 2 — 1/b; hence N(3) > 12. Robson’s methods imply that N(3) < 145, and 
2< N(4) <24] 


40. Robson has proved that N(2") < 1 + r, by using the following strategy: Allocate 
to each block of size k, where 2™ < k < 2™*1 the first available block of k locations 
starting at a multiple of 2%”. 

Let N({b1,b2,...,bn}) denote the multiplicative factor when all block sizes are 
constrained to lie in the set {b1, b2, ..., bn}, so that N(n) = N({1,2,...,n}). Robson 
and S. Krogdahl have discovered that N({b1, b2,...,bn}) = n — (b1/b2 +--+ + bn—1/bn) 
whenever b; is a multiple of b;-1 for 1 < i < n; indeed, Robson has established the 
exact formula N(2"m, {1,2,4,...,2”}) = 2"m(14 ir) 2” + 1. Thus in particular, 
N(n) > 1+ ġllgnj. He also has derived the upper bound N(n) < 1.1825lnn + 
O(1), and he conjectures tentatively that N(n) = Hn. This conjecture would follow 
if N({b1,b2,...,bn}) were equal to n — (b1/b2 + --- + bn—1/bn) in general, but this is 
unfortunately not the case since Robson has proved that N({3,4}) > 14. (See Inf. 
Proc. Letters 2 (1973), 96-97; JACM 21 (1974), 491—499.) 
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41. Consider maintaining the blocks of size 2": The requests for sizes 1, 2, 4,..., 2571 
will periodically call for a new block of size 2" to be split, or a block of that size will 
be returned. We can prove by induction on k that the total storage consumed by such 
split blocks never exceeds kn; for after every request to split a block of size 2**1, we are 
using at most kn locations in split 2*-blocks and at most n locations in unsplit ones. 

This argument can be strengthened to show that arn cells suffice, where ag = 1 
and ap = 1 + ax—1(1 — 27"); we have 


k= 0 1 2 3 4 5 
1 1 55 697 18535 
ak = 1 13 25 264 3 1024 432768 


Conversely for r < 5 it can be shown that a buddy system sometimes requires as many 
as arn cells, if the mechanism of steps R1 and R2 is modified to choose the worst 
possible available 2’-block to split instead of the first such block. 

Robson’s proof that N(2") < 1 +r (see exercise 40) is easily modified to show 
that such a “leftmost” strategy will never need more than (1 + ar)n cells to allocate 
space for blocks of sizes 1, 2, 4, ..., 2", since blocks of size 2” will never be placed in 
locations > (1+ sk) n. Although his algorithm seems very much like the buddy system, 
it turns out that no buddy system will be this good, even if we modify steps R1 and 
R2 to choose the best possible available 2/-block to split. For example, consider the 
following sequence of “snapshots” of the memory, for n = 16 and r = 3: 


11111111 11111111 00000000 00000000 
10101010 10101010 2-2-2-2- 00000000 
11110000 11110000 2-110000 00000000 
11111111 11110000 11110000 00000000 
10101010 10102-2- 10102-2- 00000000 
10001000 10002-00 10002-00 4---4--- 
10000000 10000000 10000000 4---0000 


Here 0 denotes an available location and k denotes the beginning of a k-block. In a 
Similar way there is a sequence of operations, whenever n is i multiple of 16, that forces 
an blocks of size 8 to be i full, and another in to be 4 z full. If n is a multiple of 
128, a subsequent request for an blocks of size 8 will require more than 2.5n memory 
cells. (The buddy system allows unwanted 1s to creep into an of the 8-blocks, since 
there are no other available 2s to be split at a crucial time; the “leftmost” algorithm 


keeps all 1s confined.) 

42. We can assume that m > 6. The main idea is to establish the occupancy pattern 

Rm—2(Fm—3R1)* at the beginning of the memory, for k = 0, 1, ..., where R; and Fj 

denote reserved and free blocks of size j. The transition from k to k + 1 begins with 
pages) > Rm-2(Fm -3Ri)* Rm-2Rm-2 

m—2(Fim— 3Ri)*~ 1 Fom-4Rm-2 

(Fm- 3Rı)"7 "Rm Rm-5RiRm-2 

(Fm-3 R1)" FmRm-5R1; 

then the commutation sequence Fm-3Rı Fm Rm-5Rı > Fm-3Rı Rm-2R2Rm-5Rı > 

Fəm-4RəRm-5Rı > RmRm-5 RıR2əRm-5Rı > FmRm-5RıFm-3Rı is used k times 

until we get Fm Rm—sRi(Fim—3R1)* 3 Fəm-5Rı(Fm- R1)" > Rm-2(Fm-3 R1) +t. 


Finally, when k gets large enough, there is an endgame that forces overflow unless the 
memory size is at least (n — 4m + 11)(m — 2); details appear in Comp. J. 20 (1977), 


—2 


> Rm-2 
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242-244. [Notice that the worst conceivable worst case, which begins with the pattern 
Fm-1RiF'm-1RiFm-1R1..., is only slightly worse than this; the next-fit strategy of 
exercise 6 can produce this pessimal pattern.] 


43. We will show that if Dı, D2, ... is any sequence of numbers such that Dı /m + 
Do/(m+1)+---+Dm/(2m—1) > 1 for all m > 1, and if Cm = Di /1+ D2/2+--- 
Dm/m, then Ner(n,m) < nCm. In particular, since 


Ea Eoaea EEE. EE ee 


=] | 
m m+i1 2m+1 2 2m—3 2m-2 2m— 1 


the constant sequence Dm = 1/ln 2 satisfies the necessary conditions. The proof is by 
induction on m. Let Nj = nC; for j > 1, and suppose that some request for a block 
of size m cannot be allocated in the leftmost Nm cells of memory. Then m > 1. For 
0 < j < m, we let N; denote the rightmost position allocated to blocks of sizes < j, 
or 0 if all reserved blocks are larger than j; by induction we have N; < N;. Furthermore 
we let Nj, be the rightmost occupied position < Nm, so that Nj, > Nm —m+1. 
Then the interval (Nj_,..Nj] contains at least [j(N; — N;—1)/(m + j —1)] occupied 
cells, since its free blocks are of size < m and its reserved blocks are of size > j. It 


/ 


follows that n — m > number of occupied cells > X}, j(Nj — Nj-1)/(m + j -1) = 
mNy},/(2m — 1) — (m — 1) Er Nj/(m +j)(m+j—-1) > mNm/(2m-— 1) —m— 
(m-1) OF NGA/(m+ 5-1) -1/(mt 5) = Xj nDj/(m+ 5-1) -—man-m, 
a contradiction. 

[This proof establishes slightly more than was asked. If we define the D’s by 
Di/m+---+ Dm/(2m — 1) = 1, then the sequence Ci, C2, ... is 1, Z, L, aa, men? 
and the result can be improved further, even in the case m = 2, as in exercise 38.] 


44. [F-1(1/N)], [F71(2/N)], ..., [F (N/N)]. 


APPENDIX A 


TABLES OF NUMERICAL QUANTITIES 


Table 1 


QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES 
AND IN ANALYSIS OF COMPUTER PROGRAMS (40 DECIMAL PLACES) 


V2 = 1.41421 35623 73095 04880 16887 24209 69807 85697— 

v3 = 1.73205 08075 68877 29352 74463 41505 87236 69428+ 

V5 = 2.23606 79774 99789 69640 91736 68731 27623 54406+ 
v10 = 3.16227 76601 68379 33199 88935 44432 71853 37196— 
V2 = 1.25992 10498 94873 16476 72106 07278 22835 05703— 

V3 = 1.44224 95703 07408 38232 16383 10780 10958 83919— 

V2 = 1.18920 71150 02721 06671 74999 70560 47591 52930— 
ln 2 = 0.69314 71805 59945 30941 72321 21458 17656 80755+ 
In3 = 1.09861 22886 68109 69139 52452 36922 52570 46475— 

In 10 = 2.30258 50929 94045 68401 79914 54684 36420 76011+ 
1/In 2 = 1.44269 50408 88963 40735 99246 81001 89213 74266+ 
1/ln 10 = 0.43429 44819 03251 82765 11289 18916 60508 22944— 
T = 3.14159 26535 89793 23846 26433 83279 50288 41972— 

1° = 7/180 = 0.01745 32925 19943 29576 92369 07684 88612 71344+ 
1/7 = 0.31830 98861 83790 67153 77675 26745 02872 40689+ 

n’ = 9.86960 44010 89358 61883 44909 99876 15113 53137— 

Ja = P(1/2) = 1.77245 38509 05516 02729 81674 83341 14518 27975+ 
T (1/3) = 2.67893 85347 07747 63365 56929 40974 67764 41287— 
T (2/3) = 1.35411 79394 26400 41694 52880 28154 51378 55193+ 
e = 2.71828 18284 59045 23536 02874 71352 66249 77572+ 

1/e = 0.36787 94411 71442 32159 55237 70161 46086 74458+ 

e? = 7.38905 60989 30650 22723 04274 60575 00781 31803+ 

y = 0.57721 56649 01532 86060 65120 90082 40243 10422— 

ln m = 1.14472 98858 49400 17414 34273 51353 05871 16473— 

ob = 1.61803 39887 49894 84820 45868 34365 63811 77203+ 

e = 1.78107 24179 90197 98523 65041 03107 17954 91696+ 
e”/4 = 2.19328 00507 38015 45655 97696 59278 73822 34616+ 
sin 1 = 0.84147 09848 07896 50665 25023 21630 29899 96226— 
cos 1 = 0.54030 23058 68139 71740 09366 07442 97660 37323+ 
—Ç' (2) = 0.93754 82543 15843 75370 25740 94567 86497 78979— 
¢(3) = 1.20205 69031 59594 28539 97381 61511 44999 07650— 
In @ = 0.48121 18250 59603 44749 77589 13424 36842 31352— 
1/In ¢ = 2.07808 69212 35027 53760 13226 06117 79576 77422— 
—InIn2 = 0.36651 29205 81664 32701 24391 58232 66946 94543— 
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Table 2 


QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES 
AND IN ANALYSIS OF COMPUTER PROGRAMS (45 OCTAL PLACES) 


The names at the left of the “=” signs are given in decimal notation. 


0.1 = 0.06314 63146 31463 14631 46314 63146 314638 14631 46315— 

0.01 = 0.00507 58412 17270 24865 60507 53412 17270 24365 60510— 

0.001 = 0.00040 61115 64570 65176 76355 44264 16254 02030 44672+ 

0.0001 = 0.00008 21556 13530 70414 54512 75170 33021 15002 35223— 
0.00001 = 0.00000 24761 32610 70664 36041 06077 17401 56063 384417— 
0.000001 = 0.00000 02061 57364 05586 66151 55323 07746 44470 260334 
0.0000001 = 0.00000 00153 27745 15274 53644 12741 72312 20354 02151+ 
0.00000001 = 0.00000 00012 57143 56106 04303 473874 77341 01512 63327 
0.000000001 = 0.00000 00001 04560 27640 46655 12262 71426 40124 217424 
0.0000000001 = 0.00000 00000 06676 83766 35367 55653 37265 34642 01627— 
2= 1.32404 74681 77167 46220 42627 66115 46725 12575 17435+ 

V3 = 1.56663 65641 30231 25163 54453 50265 60361 84073 42223— 

V5 = 2.17067 36834 57722 47602 51471 63008 00563 55620 32021— 

V10 = 3.12305 40726 64555 22444 02242 57101 41466 33115 225324 

V2 = 1.20505 05746 15345 05342 10756 65334 25574 22415 03024+ 

V3 = 1.84233 50444 22175 13134 67363 16133 05334 31147 60121— 

V2 = 1.14067 74050 61556 12455 72152 64430 60271 02755 73136+ 

In2= 0.54271 02775 75071 73632 57117 07316 30007 71366 53640+ 

In3= 1.06237 24752 55006 05227 32440 63065 25012 35574 553374 

ln10 = 2.23278 067385 52524 25405 56512 66542 56026 46050 50705+ 
1/m2= 1.84252 16624 58405 77027 85750 87766 40644 35175 0438538+ 
1/n10 = 0.83626 75425 11562 41614 52325 33525 27655 14756 06220— 

m= 3.11087 55242 10264 30215 14230 68050 56006 70163 211224 

1° = 7/180 = 0.01073 72152 11224 72344 25603 54276 63351 22056 115444 
1/m = 0.24276 30155 62344 20251 238760 47257 50765 15156 70067— 

n’? = 11.67517 14467 62135 713822 25561 15466 30021 40654 34103— 

yT =I (1/2) = 1.61837 61106 64736 65247 47035 40510 15273 34470 17762— 
) = 2.58847 385234 51013 61316 73106 47644 54653 00106 66046— 

T (2/3)= 1.26523 57112 14154 74812 54572 87655 60126 23231 02452+ 
e = 2.55760 52130 50585 51246 52773 42542 00471 72363 61661+ 

1/e = 0.27426 538066 13167 46761 52726 75486 02440 52371 083554 

e = 7.30714 45615 23355 33460 63507 35040 32664 25356 502174 

Y= O44742 14770 67666 06172 23215 74376 01002 51313 25521 

lnm = 1.11206 40443 47503 86413 65374 52661 52410 87511 460574 
b= 1.47433 57156 27751 238701 27634 71401 40271 66710 15010+ 

e” = 1.61772 13452 61152 65761 22477 386553 59927 17554 21260+ 

= 2.14275 31512 16162 52370 35530 11342 53525 44307 02171— 
sinl = 0.65665 24436 04414 73402 03067 23644 11612 O7474 14505— 
cosl= 0.42450 50037 32406 42711 07022 14666 27820 70675 12321+ 
—C'(2) = 0.74001 45144 582538 42362 42107 23350 50074 46100 27706+ 
¢(3) = 1.147385 00023 60014 20470 15613 42561 31715 10177 06614+ 
In¢d= 0.36630 26256 61213 01145 13700 41004 52264 30700 40646+ 
1/lnġ = 2.04776 60111 17144 41512 11486 16575 00355 43630 40651+ 
—InIn2= 0.27351 71233 67265 63650 17401 56637 26334 31455 57005— 
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Several of the 40-digit values in Table 1 were computed on a desk calculator 
by John W. Wrench, Jr., for the first edition of this book. When computer 
software for such calculations became available during the 1970s, all of his 
contributions proved to be correct. See the answer to exercise 1.3.3-23 for the 
40-digit value of another fundamental constant. 


Table 3 


VALUES OF HARMONIC NUMBERS, BERNOULLI NUMBERS, 
AND FIBONACCI NUMBERS, FOR SMALL VALUES OF n 


n Ay Bn Fn n 
0 0 1 0 0 
1 1 —1/2 1 1 
2 3/2 1/6 1 2 
3 11/6 0 2 3 
4 25/12 —1/30 3 4 
5 137/60 0 5 5 
6 49/20 1/42 8 6 
7 363/140 0 13 7 
8 761/280 —1/30 21 8 
9 7129/2520 0 34 9 
10 7381/2520 5/66 55 10 
11 83711/27720 0 89 11 
12 86021/27720 —691/2730 144 12 
13 1145993/360360 0 233 13 
14 1171733/360360 7/6 377 14 
15 1195757/360360 0 610 15 
16 2436559/720720 —3617/510 987 16 
17 42142223 /12252240 0 1597 17 
18 14274301/4084080 43867 /798 2584 18 
19 275295799/77597520 0 4181 19 
20 55835135/15519504 —174611/330 6765 20 
21 18858053/5173168 0 10946 21 
22 19093197/5173168 854513/138 17711 22 
23 444316699/118982864 0 28657 23 
24 1347822955 /356948592 —236364091 /2730 46368 24 
25 34052522467/8923714800 0 75025 25 
26 34395742267 /8923714800 8553103 /6 121393 26 
27 312536252003/80313433200 0 196418 27 
28 315404588903/80313433200 —23749461029/870 317811 28 
29 9227046511387/2329089562800 0 514229 29 
30 9304682830147 /2329089562800 8615841276005 /14322 832040 30 
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1 
n+2x 


1 
For any x, let Hy, = (= = 
n 


nal 


Hız =2— 2ln2, 


) . Then 


Hija = 3 — §r/vV3 — ł n3, 

Haja = $ + }r/ V3 — $ n3, 

Hija =4— 57 —3ln2, 

Hzj4 = $ + in — 31n2, 

Hijs =5 — 069/25-1/4 — 3 n5 — iv5lno, 
Hays = Š — Ang-3/25-1/4 — Š In5 + 1V5In g, 
Haps = $+ $no-97?5-1/4 — $ In5 + bV5In 9, 
Hajs = $ + 4ng8??5-1/4 5m5 — 1 VBIng, 
Hije = 6 — 403 — 2ln 2 — 31n3, 

Aso = Ê + $nV3 — 21n 2 — 31n3, 


and, in general, when 0 < p < q (see exercise 1.2.9-19), 
2 
Hp = f 5 cot om ln2q +2 5 cos Z” r- lnsin “7. 
7 q l<n<q/2 q 
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INDEX TO NOTATIONS 


In the following formulas, letters that are not further qualified have the following 
significance: 


j,k integer-valued arithmetic expression 
m,n nonnegative integer-valued arithmetic expression 
T, Y real-valued arithmetic expression 
f real-valued or complex-valued function 
P pointer-valued expression (either A or a computer address) 
S,T set or multiset 
a string of symbols 
Where 
Formal symbolism Meaning defined 
V « E | give variable V the value of expression E Lf 


U & V | interchange the values of variables U and V | 1.1 

An or Ajn] | the nth element of linear array A ii 
Amn or Afm, n] | the element in row m and column n of rect- 
angular array A ti 
NODE (P) | the node (group of variables that are indi- 
vidually distinguished by their field names) 
whose address is P, assuming that P 4 A 21 
F(P) | the variable in NODE (P) whose field name is F | 2.1 
CONTENTS (P) | contents of computer word whose addressisP | 2.1 


LOC(V) | address of variable V within a computer 2.1 
P < AVAIL | set the value of pointer variable P to the 
address of a new node 223 
AVAIL =P | return NODE(P) to free storage; all its fields 
lose their identity 2.2.3 
top(S) | node at the top of a nonempty stack S Zid 
X <5 | pop up S to X: set X + top(S); then delete 
top(S) from nonempty stack S 2.21 


S < X | push down X onto S: insert the value X as 
a new entry on top of stack S 220k 
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Where 
Formal symbolism Meaning defined 
(R? a: b) | conditional expression: denotes 
a if relation R is true, b if R is false 
[R] | characteristic function of relation R: 
(R? 1: 0) 1.23 
ôkj | Kronecker delta: [j = k] 23 
[z”] g(z) | coefficient of z” in power series g(z) 1.2.9 
XC f(k) | sum ofall f(k) such that the variable k is an 
R(k) integer and relation R(k) is true pie get 
II f(k) | product of all f(k) such that the variable k 
R(k) is an integer and relation R(k) is true 1.2.3 
min f(k) | minimum value of all f(k) such that the var- 
R(R) iable k is an integer and relation R(k) is true | 1.2.3 
max f(k) | maximum value of all f(k) such that the var- 
a iable k is an integer and relation R(k) is true | 1.2.3 
j\k | j divides k: k mod j = 0 and j > 0 1.24 
S\T | set difference: {a | a in S and a not in T} 
gcd(j,k) | greatest common divisor of j and k: 
(j=k=07 0: max a) iá 
d\j, d\k 
jL k | j is relatively prime to k: gcd(j,k) = 1 124 
AT | transpose of rectangular array A: 
AT [j,k] = Afk, j] 
a® | left-right reversal of a 
x” | x to the y power (when z is positive) 122 
x? | x to the kth power: 
(: >o? ff «: T i22 
0<j<k 
z? | x to the k rising: T(x + k)/I(2) = 
(i >0 [[ (+a: 1/(@+t =) 1.2.5 
O<j<k 
xE | x to the k falling: x!/(£ — k)! = 
(i >0? [J @-3: IW/(@- e=) 1.2.5 


O0<j<k 
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Where 
Formal symbolism Meaning defined 
n! | n factorial: T(n + 1) = ne 12.5 
(;) binomial coefficient: (k < 0? 0: x£/k!) 126 
( n ) multinomial coefficient (defined only when 
N1,N2,-..,Nm n = ni +n +: Nm) 1.2.6 
| Stirling number of the first kind: 
5 kika... kn-m 126 
0<kı<k2<--<kn-m<n 
{"} Stirling number of the second kind: 
kiko... kn-m 1.2.6 
1<k1LSk2<<kn-m<m 
{a | R(a)} | set of all a such that the relation R(a) is true 
{a1,..., an} | the set or multiset {ap |1 <k <n} 
{x} | fractional part (used in contexts where a 
real value, not a set, is implied): x — |x| 1.2:11.2 
a, +a2 +: +an | n-fold sum: Ya a; 1.2.3 
[a..b] | closed interval: {x |a < x < b} 122 
(a..b) | open interval: {x | a < x < b} 1.2.2 
[a..b) | half-open interval: {x |a < x < b} 122 
(a..b] | half-closed interval: {x |a < x < b} 1.2.2 
|S| | cardinality: the number of elements in set S$ 
|x| | absolute value of x: (x > 0? a: — x) 
la| | length of a 
|x| | floor of x, greatest integer function: max,<,k| 1.2.4 
[x] | ceiling of x, least integer function: ming>,k | 1.2.4 
zmod y | mod function: (y = 0? x: x — y|z/y]) 1.2.4 
x = x' (modulo y) | relation of congruence: x mod y = a’ mod y 1.2.4 
O(f(n)) | big-oh of f(n), as the variable n + co LALI 
O(f(z)) | big-oh of f(z), as the variable z > 0 FALI 
Q(f(n)) | big-omega of f(n), as the variable n — co LILI 
O(f(n)) | big-theta of f(n), as the variable n — oo 12.111 
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Where 
Formal symbolism Meaning defined 
log, x | logarithm, base b, of x (when x > 0, 
b> 0, and b Æ 1): the y such that z = bY 122 
Ing | natural logarithm: log, x 1:2:2 
lgx | binary logarithm: log, x 12.2 
exp x | exponential of x: e” 1.2.9 
(Xn) | the infinite sequence Xo, X1, Xo, ... 
(here the letter n is part of the symbolism) 1.2.9 
f'(x) | derivative of f at x 1.2.9 
f(a) | second derivative of f at x 1.2.10 
f(x) | nth derivative: (n = 0? f(x): g'(x)), 
where g(x) = f-Y(a) | 1.2.11.2 
HC’ | harmonic number of order z: 5 1/k” Lr 
1<k<n 
A, | harmonic number: HY) LF 
F,, | Fibonacci number: 
(n <1? n: Fa-1 + Fr—2) 1.2.8 
Bn | Bernoulli number: n! [z”] z/(e* — 1) 12112 
det(A) | determinant of square matrix A 1.2:3 
sign(x) | sign of x: [x >0] — [xr <0] 
¢(x) | zeta function: limyn—oo HE? (when x > 1) 1.2.7 
I(x) | gamma function: (x — 1)! = y(x, 00) 1.2.5 
7(x,y) | incomplete gamma function: fy e~'t?~! dt Liia 
y | Euler’s constant: limn—o(Hn — Inn) Loe 
e | base of natural logarithms: >7,,9 1/n! 122 
m | circle ratio: 4>7,59(-1)"/(2n + 1) 1.2.2 
co | infinity: larger than any number 
A | null link (pointer to no address) 21 
€ | empty string (string of length zero) 
Ø | empty set (set with no elements) 
@ | golden ratio: (1 + v5) 1.2.8 
y(n) | Euler’s totient function: 5 [k Ln] 1.2.4 
0<k<n 
xz 7y | xis approximately equal to y 12.5 
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Where 
Formal symbolism Meaning defined 
Pr(S(X)) | probability that statement S(X) is true, for 
random values of X 122.10 
EX | expected value of X: 0.x Pr(X = x) 1.2.10 
mean(g) | mean value of the probability distribution 
represented by generating function g: g’(1) 1.2.10 
var(g) | variance of the probability distribution 
represented by generating function g: 
g (1) +g) — 9 (1)? 1.2.10 
(min 21, ave 2a, a random variable having minimum 
max 23, dev x4) | value 21, average (expected) value x2, 
maximum value x3, standard deviation x4 1:23:10 
P* | address of preorder successor of NODE (P) in 
a binary tree or tree Boab, 2.008 
Ps | address of inorder successor of NODE(P) in a 
binary tree, postorder successor in a tree PAE RIA 
Pt | address of postorder successor of NODE (P) in 
a binary tree 2al 
*P | address of preorder predecessor of NODE (P) 
in a binary tree or tree 2al, 23:2 
$P | address of inorder predecessor of NODE (P) in 
a binary tree, postorder predecessor in a tree | 2.3.1, 2.3.2 
tP | address of postorder predecessor of NODE (P) 
in a binary tree 2:31 
Į | end of algorithm, program, or proof LA 
u | one blank space Lot 
rA | register A (accumulator) of MIX 1.3.1 
rX | register X (extension) of MIX Lal 
rIl,...,rI6 | (index) registers I1, ..., I6 of MIX La 
rJ | (jump) register J of MIX iad 
(L:R) | partial field of MIX word,0<L<R<5 13.1 
OP ADDRESS,I(F) | notation for MIX instruction LLL 
u | unit of time in MIX Last 
* | “self” in MIXAL Le 
OF, 1F, 2F,..., 9F | “forward” local symbol in MIXAL 1:3:2 
OB, 1B, 2B, ..., 9B | “backward” local symbol in MIXAL Lei 
OH, 1H, 2H, ..., 9H | “here” local symbol in MIXAL 1:32 
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Algorithm 1.1F, 2, 4. 
Algorithm 1.1F, 466. 
Algorithm 1.2.1E, 13-14. 
Algorithm 1.2.11, 11-12. 
Algorithm 1.2.2E, 470. 
Algorithm 1.2.2L, 26. 
Law 1.2.4A, 40. 

Law 1.2.4B, 40. 

Law 1.2.4C, 40. 

Law 1.2.4D, 40. 

Theorem 1.2.4F, 41. 
Theorem 1.2.7A, 77. 
Theorem 1.2.8A, 81-82. 
Theorem 1.2.10A, 101. 
Algorithm 1.2.10M, 96. 
Program 1.2.10M, 145, 186. 
Theorem 1.2.11.3A, 119. 
Algorithm 1.3.2E, 160. 
Program 1.3.2E, 516-518. 
Program 1.3.2M, 145. 
Algorithm 1.3.2P, 147. 
Program 1.3.2P, 148-149. 
Algorithm 1.3.3A, 166-167. 
Program 1.3.3A, 168-169. 
Algorithm 1.3.3B, 173. 
Program 1.3.3B, 174-175. 
Algorithm 1.3.31, 176. 
Program 1.3.31, 177. 
Algorithm 1.3.3J, 177. 
Program 1.3.3J, 178. 


Program 1.4.3.1M, 204-211, 530. 


Algorithm 1.4.4A, 221-222. 
Program 1.4.4A, 222. 
Algorithm 1.4.4A’, 534. 
Algorithm 1.4.4B, 222. 
Program 1.4.4B, 223. 
Algorithm 1.4.4B’, 534. 
Algorithm 1.4.4R, 222. 


Program 1.4.4R, 223. 
Algorithm 1.4.4R’, 534. 
Algorithm 2.1A, 235. 
Program 2.1A, 236. 
Algorithm 2.1B, 235. 
Program 2.1B, 535. 
Algorithm 2.1C, 535. 
Algorithm 2.1D, 535. 
Algorithm 2.2.2A, 541. 
Algorithm 2.2.2G, 248-249. 
Algorithm 2.2.2R, 249-250. 
Algorithm 2.2.3A, 549. 
Program 2.2.3A, 550. 
Algorithm 2.2.3B, 549-550. 
Program 2.2.3B, 550. 
Algorithm 2.2.31, 546. 
Program 2.2.31, 546. 
Algorithm 2.2.3T, 265, 548. 
Program 2.2.3T, 266-268, 
548-549. 
Algorithm 2.2.4A, 276. 
Program 2.2.4A, 277-278. 
Algorithm 2.2.4M, 277. 
Program 2.2.4M, 552-553. 
Program 2.2.5D, 555. 
Subroutine 2.2.5D, 287. 
Coroutine 2.2.5E, 284-285. 
Program 2.2.5E, 293-295, 
554-555. 
Coroutine 2.2.5U, 283-284. 
Program 2.2.5U, 292-293. 
Algorithm 2.2.6G, 557. 
Algorithm 2.2.68, 304-305. 
Program 2.2.65, 557-559. 
Algorithm 2.3.1A, 571. 
Theorem 2.3.1A, 328-329. 
Algorithm 2.3.1C, 329-330, 
570. 
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Program 2.3.1C, 572-573. 
Algorithm 2.3.1F, 566. 
Algorithm 2.3.11, 327. 
Algorithm 2.3.1L, 570. 
Lemma 2.3.1P, 329. 
Algorithm 2.3.1R, 569. 
Algorithm 2.3.15, 323. 
Program 2.3.15, 325. 
Algorithm 2.3.1T, 320. 
Program 2.3.1T, 325, 567. 
Algorithm 2.3.1T’, 565. 
Algorithm 2.3.1U, 567. 
Program 2.3.1U, 568. 
Algorithm 2.3.1W, 567—568. 
Algorithm 2.3.1X, 570. 
Algorithm 2.3.2D, 340. 
Program 2.3.2D, 342-345, 
573-574. 
Algorithm 2.3.3A, 357-359. 
Algorithm 2.3.3E, 354-355. 
Algorithm 2.3.3F, 351. 
Algorithm 2.3.3G, 577. 
Algorithm 2.3.3L, 575. 
Algorithm 2.3.3P, 577. 
Algorithm 2.3.3T, 576. 
Theorem 2.3.4.1A, 363-364. 
Theorem 2.3.4.1K, 368-369. 
Algorithm 2.3.4.2C, 581. 
Theorem 2.3.4.2D, 376. 
Lemma 2.3.4.2E, 375. 
Algorithm 2.3.4.2G, 581. 
Theorem 2.3.4.2G, 375. 
Theorem 2.3.4.3K, 382. 
Algorithm 2.3.4.4D, 590. 
Theorem 2.3.4.4R, 394-395. 
Algorithm 2.3.4.5H, 596. 
Algorithm 2.3.5A, 415. 
Algorithm 2.3.5A’, 415. 


Algorithm 2.3.5B, 415-416. 


Algorithm 2.3.5C, 416. 
Algorithm 2.3.5D, 417. 


Algorithm 2.3.5E, 418-419. 


Program 2.3.5E, 601-602. 
Algorithm 2.3.5F, 603. 


Algorithm 2.3.5G, 603-604. 


Algorithm 2.3.5H, 605. 
Algorithm 2.3.55, 602-603. 
Algorithm 2.3.5Z, 605. 
Algorithm 2.4A, 428-429. 
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Algorithm 2.4A’, 605. 
Algorithm 2.4B, 429-430. 
Algorithm 2.4B’, 605. 
Algorithm 2.4B”, 606. 
Algorithm 2.4C, 431. 
Algorithm 2.4C’, 606-607. 
Algorithm 2.5A, 437. 
Program 2.5A, 607. 
Algorithm 2.5A’, 438. 
Algorithm 2.5A”, 608. 


Algorithm 2.5A*, 608-610. 


Program 2.5A*, 609. 
Algorithm 2.5B, 440. 
Algorithm 2.5C, 441-442. 
Program 2.5C, 610. 
Algorithm 2.5G, 613-614. 
Program 2.5G, 614-615. 
Algorithm 2.5P, 445. 
Algorithm 2.5R, 443. 
Program 2.5R, 611-612. 
Algorithm 2.55, 443-444. 
Program 2.55, 612-613. 


Numerical experimentations are necessary 
to fully understand the algorithms and theorems in this book. 


— STEPHANE MALLAT, A Wavelet Tour of Signal Processing (1998) 


INDEX AND GLOSSARY 


Some Men pretend to understand a Book 


by scouting thro’ the Index: 


as if a Traveller should go about to describe a Palace 


when he had seen nothing but the Privy. 


— JONATHAN SWIFT, Mechanical Operation of the Spirit (1704) 


When an index entry refers to a page containing a relevant exercise, see also the answer to 
that exercise for further information. An answer page is not indexed here unless it refers to a 
topic not included in the statement of the exercise. 


(), 164, see Identity permutation. 
Q-2-trees, 317. 
oriented, 398. 
0-origin indexing, 254, 282, 299-301, 
305-306. 
2-d trees, 564. 
y (Euler’s constant), 75, 107, 114, 619-620. 
m (circle ratio), 21, 619-620. 
s “random” example, 397. 
Wallis’s product for, 52, 116. 
¢ (golden ratio), 13, 18, 21, 80, 83-86, 
619-620. 


A-register of MIX, 125. 


Aardenne-Ehrenfest, Tatyana van, 375, 379. 


Aarons, Roger Michael, 528. 
Abel, Niels Henrik, 58, 498. 
binomial theorem, 58, 71—73, 398. 
limit theorem, 95, 588. 
Absolute error, 116. 
Absolute value, 21. 
Absolutely convergent series, 29. 
ACE computer, 193, 229. 
Pilot, 230. 
Acerbi, Fabio, 593. 
Adams, Charles William, 230. 
ADD, 131-132, 208. 
Add to list: see Insertion. 


Addition of polynomials, 275-280, 357-359. 


Address: A number used to identify a 
position in memory. 
field of MIXAL line, 145, 151-153. 
of node, 233. 
portion of MIX instruction, 127. 
Address transfer operators of MIX, 
133-134, 210. 
Adjacent vertices of a graph, 363. 
Adobe Systems, 202. 
Aebly, Jakob, 536. 
Agenda, 288, 291, 296, see Priority queue. 
Aho, Alfred Vaino, 560. 
Ahrens, Wilhelm Ernst Martin Georg, 162. 
Alhazen, see Ibn al-Haytham. 
al-Khwarizmi, Abu ‘Abd Allah 
Muhammad ibn Musa 
(Ll ase Ge tone alll se sil), 
1, 79. 


ALF (alphabetic data), 151, 152, 155. 

Algebraic formulas, 313. 
differentiation, 90, 338-347, 459. 
manipulation of, 459—462. 
representation as trees, 337, 459. 
simplification of, 339, 347. 

ALGOL language, viii, 202, 229. 

Algorithm, origin of word, 1-2. 

Algorithms, 1-9. 

analysis of, vi, 7, 96-107, 170-172, 179, 
250, 253, 268, 278-280, 324-326, 331, 
380-382, 444—445, 451. 

communication of, 16. 

effective, 6, 8, 9. 

equivalence between, 467. 

form of in this book, 2—4. 

hardware-oriented, 26, 252, 611. 

how to read, 4, 16. 

proof of, 5-6, 13-17, 321, 361, 422, 434. 

properties of, 4-6, 9. 

random paths in, 380-381. 

set-theoretic definition, 7—9. 

theory of, 7, 9. 

Allocation of tables, see Dynamic storage 
allocation, Linked allocation, 
Representation, Sequential allocation. 

Alpern, Steven Robert, 526. 

Alphameric character: A letter, digit, or 
special character symbol. 

codes for MIX, 136, 138, 140-141. 

AMM: American Mathematical Monthly, 
published by the Mathematical 
Association of America since 1894. 

Amortized running time, 254. 

Analysis of algorithms, vi, 7, 96-107, 
170-172, 179, 250, 253, 268, 278-280, 
324-326, 331, 380-382, 444—445, 451. 

Analytical Engine, 1, 229. 

Ancestor, in a tree structure, 311, 348, 562. 

Anticipated input, 216, see Buffering. 

Antisymmetric relation, 261. 

Apostol, Tom Mike, 28. 

Arabic mathematics, 1, 162. 

Araújo, Saulo, 565. 

Arbogast, Louis François Antoine, 52, 105. 

Arborescences, 363, see Oriented trees. 

Arc digraph, 379. 

Arc in a directed graph, 372. 
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Area of memory, 435. 

Arguments of subroutines, 187, 189. 

Arithmetic: Addition, subtraction, 
multiplication, and division, vi, ix. 

fixed point, 158. 

floating point, 131, 306. 

operators of MIX, 131-133, 208. 
polynomial, 275-280, 357-359, 361. 
scaled decimal, 160-161. 

Arithmetic expressions, see Algebraic 
formulas. 

Arithmetic progression, sum of, 11, 
31-32, 56. 

Array: A table that usually has a 
k-dimensional rectangular structure, 
4, 232, 298-307. 

linked, 301-307. 

one-dimensional, see Linear list. 

represented as a tree, 315. 

sequential allocation, 159, 299-301, 
305-307. 

tetrahedral, 300-301, 306. 

two-dimensional, see Matrix. 

uninitialized, 307. 

Arrows, used to represent links in 
diagrams, 234. 

Assembly language: A language that is 
intended to facilitate the construction 
of programs in machine language by 
making use of symbolic and mnemonic 
conventions to denote machine 
language instructions. 

contrasted with high-level language, 
236, 535. 
for MIX, 144-157. 

Assembly program, 145, 153. 

Assertions, inductive, 15—20. 

Assigning a buffer, 219-223, 226-227. 

Assignment operation (+), 3. 

Asterisk (“*”) in assembly language, 
146, 149, 153. 

Asymmetric relations, 261. 

Asymptotic values: Functions that express 
the limiting behavior approached by 
numerical quantities. 

derivation of, 107-123, 243, 396-397, 
525, 565. 
Atom (in a List), 315, 408-412, 418. 
purpose of, 410. 

Automata theory, 230, 240, 463-464. 

Automaton: An abstract machine that is 
defined formally, often intended to 
be a model of some aspects of actual 
computers (plural: Automata), 463-464. 

AVAIL stack: Available space list, 256. 

Available space list, 256-261, 266, 269, 
278, 291, 413-414, 435-444. 

history, 458. 
variable-size blocks, 435—456. 

Average value of a random variable, 

97-98, 103. 
from a generating function, 100-103. 


INDEX AND GLOSSARY 631 


Babbage, Charles, 1, 229. 
Bachmann, Paul Gustav Heinrich, 107. 
Backus, John Warner, 230. 
Bailey, Michael John, 462. 
Baker, Henry Givens, Jr., 605. 
Balanced directed graph, 374-375. 
Ball, Thomas Jaudon, 369. 
Ball, Walter William Rouse, 162. 
Ballot problem, 536-537. 
Barnes, Ernest William, 507. 
Barnett, Michael Peter, 462. 
Barrington (= Mix Barrington), David 
Arno, 526. 
Barry, David McAlister, subtle reference 
to, xi, 274. 
Barton, David Elliott, 66, 536. 
Base address, 244. 
Bead, 233, see Node. 
Before and after diagrams, 260-261, 
28, 281,571. 
Beigel, Richard, 467. 
Bell, Eric Temple, 87. 
Bell Interpretive System, 230. 
Bellman, Richard Ernest, xv. 
Bendix G20, 124. 
Bennett, John Makepeace, 230. 
Berger, Robert, 385. 
Bergeron, François, 395. 
Bergman, George Mark, 495, 593. 
Berlekamp, Elwyn Ralph, 273. 
Berman, Martin Fredric, 523. 
Berners-Lee, Mary Lee née Woods, 230. 
Bernoulli, Daniel, 83. 
Bernoulli, Jacques (= Jakob = James), 
112. 115; 
numbers Bn, 76, 91, 112-115. 
numbers, table, 621. 
polynomials, 44, 113-115, 503. 
Bernoulli, Jean (= Johann = John), III, 495. 
Bernstein, Sergei Natanovich (Bepnmrein, 
Cepreit Haranosns), 104. 
Bertrand, Joseph Louis François, 
postulate, 512. 
Berztiss, Alfs Teodors, 462. 
Best-fit method of storage allocation, 
436-437, 447, 453-456. 
Beta function B(x, y), 72. 
Bhaskara II, Acarya, son of Mahesvara 
(ATER, qa), 54. 
Bienaymé, Irénée Jules, 98. 
Bienstock, Daniel, 514. 
Big-Oh notation, 107—111, 118. 
Big-Omega notation, 110-111. 
Big-Theta notation, 110. 
Bigelow, Richard Henry, 562. 
Binary computer: A computer that 
manipulates numbers primarily in the 
binary (radix 2) number system. 
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Binary logarithms, 23, 26. 
Binary number system, 24-26. 
Binary operator: A function of two 
variables, 337. 
Binary trees, 312, 317, 318-337, 363, 459. 
complete, 401, 405, 563. 
copying of, 329-330, 333, 347. 
correspondence to trees and forests, 
334-335, 346. 
definition of, 312. 
“Dewey” notation for, 317, 331, 346, 405. 
enumeration of, 388-389, 405. 
equivalent, 328-329. 
erasing of, 333. 
extended, 399-406. 
linked, 318. 
oriented, 396-397. 
path length of, 399—406. 
representation of, 318, 322, 327, 333-334. 
right-threaded, 327, 332-334. 
sequential, 401. 
similar, 327-329. 
threaded, 322, 331-332, 460. 
traversal of, 319-320, 323, 331-332, 
459-460. 
with shared subtrees, 326, 603. 
Binet, Jacques Philippe Marie, 36, 39, 
407, 475, 582. 
Binomial coefficients, 52—74, 89. 
asymptotic values, 72. 
bounds on, 74. 
combinatorial interpretation, 52-53, 73. 
defined, 53. 
generalized, 65, 72, 85. 
generating functions, 90, 94. 
history, 53-54. 
inequalities involving, 74. 
sums involving, 56-74, 76-78, 85, 96. 
table, 54. 
Binomial distribution, 101-102. 
tail, 106. 
Binomial number system, see Combinatorial 
number system. 
Binomial theorem, 57—58, 90. 
Abel’s generalization, 58, 71-73, 398. 
generalizations of, 64, 70-73, 90, 
398-399, 488. 
Hurwitz’s generalization, 399, 488. 
Binomial tree, xx. 
Bipartite trees, 398. 
Bit: “Binary digit’? either zero or unity. 
BIT: Nordisk Tidskrift for Informations- 
Behandling, an international journal 
published in Scandinavia since 1961. 
Bitwise operations, 442, 455, 510, 553. 
Blaauw, Gerrit Anne, 458. 
Blikle, Andrzej Jacek, 329. 
Block of external data, 136-137. 
Block of memory, 435. 
Blocking of records, 218, 225. 


Bobrow, Daniel Gureasko, 421, 460, 461. 

Boles, David Alan, 452. 

Bolzano, Bernard Placidus Johann 
Nepomuk, 382. 

Boncompagni, Prince Baldassarre, 79. 

Boolean functions, 526-527. 

Boothroyd, John, 177. 

Bootstrapping, 143. 

Borchardt, Carl Wilhelm, 406, 583. 

Bottom of stack, 241. 

Bottom-up process, 309, 351. 

Boundary tag method of storage allocation, 
440-442, 453-454, 461. 

Bracket notation for coefficients, 92. 

Bracket notation for logical statements, 
32-33, 471, see Iverson’s convention. 

Branch instruction: A conditional 
“jump” instruction. 

Branch node of tree, 308. 

Breadth-first search, 351. 

Brenner, Norman Mitchell, 523. 

Brent, Richard Peirce, 563. 

Briggs, Henry, 26. 

Broline, Duane Marvin, 601. 

Brother, in a tree structure, 311. 

Brouwer, Luitzen Egbertus Jan, 406. 

Bruijn, Nicolaas Govert de, 121, 122, 375, 
379, 380, 478, 504, 543, 565. 

Buddy system for storage allocation, 
442-444, 447-448, 454-456, 461. 
Buffering of input-output, 158, 216-228. 

history, 231. 
swapping, 147, 159, 217-218, 225. 
Bugs: Errors or defects; see Debugging. 
Burke, John, 310. 
Burks, Arthur Walter, 359. 
Burleson, Peter Barrus, 462. 
Burroughs B220, 124. 
Burroughs B5000, 461. 
Busche, Conrad Heinrich Edmund 
Friedrich, 43. 
Busy waiting, 216. 
Byte: Basic unit of data, usually associated 
with alphameric characters, 125. 
in MIX, 124-125, 139. 
Byte size in MIX: The number of distinct 
values that might be stored in a byte. 


C language, 233, 556. 
Cache, 528. 
CACM: Communications of the ACM, 
a publication of the Association for 
Computing Machinery since 1958. 
Cajori, Florian, 24. 
Calendar, 160. 
California Institute of Technology, x, 282. 
Call: To activate another routine in 
a program. 
Calling sequence, 187—190, 193, 196-197. 
Campbell, John Arthur, 450. 


Canonical cycle notation for permutations, 
178-179. 

Canonical representation of oriented trees, 
390-394, 397-398, 590-591. 

Capelli, Alfredo, 50. 

Car: LISP terminology for the first 
component of a List; analogous to 
INFO and DLINK on page 411, or to 
ALINK on page 415. 

Cards, playing, 51, 69, 233-237, 238, 
377-378. 

Cards, punched, 136-137, 152, 229. 

Carlitz, Leonard, 96, 499, 506. 

Carlyle, Thomas, xiv. 

Carpenter, Brian Edward, 229. 

Carr, John Weber, III, 458. 

Carroll, Lewis (= Dodgson, Charles 
Lutwidge), 465. 

Case Institute of Technology, i. 

Cassini, Gian (= Giovanni = Jean) 
Domenico (= Dominique), 81. 

Catalan, Eugéne Charles, 407. 

numbers, 389, 407, 536. 
Cate, Esko George, 523. 
Cauchy, Augustin Louis, 39, 92, 475, 
490, 506, 520, 582. 
inequality: 0D apb)? < (= CALD b2), 
36. 
matrix, 37—38, 475. 

Cayley, Arthur, 396, 406-407, 586, 597. 

CDC 1604 computer, 124, 529. 

Cdr: LISP terminology for the remainder of 
a List with its first component deleted; 
analogous to RLINK on page 411, or 
to BLINK on page 415. 

Ceiling function [x], 39, 41. 

Cell: A word of the computer memory, 127. 

Cellar, 240. 

Central moment of a probability 
distribution, 105. 

Centroid of a free tree, 387-388, 397, 589. 

Chain: A word used by some authors to 
denote a linked linear list, by others 
to denote a linearly ordered set. 

Chain rule for differentiation, 52. 

Chaining: A word used by some authors 
in place of “linking” 

Chakravarti, Gurugovinda (88TA 
Daas), 53. 

Change of summation variable, 28, 32-33. 

Channel: A data-transmission device 
connected to a computer, 224. 

CHAR (convert to characters), 138. 

Character codes of MIX, 136, 138, 140-141. 

Characteristic function of a probability 
distribution, 103. 

Characteristic polynomial of a matrix, 499. 

Charles Philip Arthur George of Edinburgh, 
Prince of Wales, 310. 

Cheam, Tat Ong, 450. 
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Cheating, 582. 

Chebyshev (= Tschebyscheff), Pafnutii 
Lvovich (Ue6nnmesn, [lacdbuytiit 
JIbpopuub = Uednues, Tlacbuytui 
JIppopu4), inequalities, 98, 104, 472. 

Checkerboard, 435—436. 

Checkerboarding, see Fragmentation. 

Chen, Tien Chi (Sf 7C#R), 471. 

Cheney, Christopher John, 421. 

Cheney, Ednah Dow Littlehale, 377. 

Chernoff, Herman, 502. 

Chess, 6, 194, 272. 

Chia Hsien (S36), 53. 

Child link, 427—433. 

Children in tree structures, 311, 317, 
334-335, 352. 

Chinese mathematics, 53, 59, 75, 407. 

Chowla, Paromita (ARASI BIST or 
great aaa), 307. 

Christian IX, King of Denmark, 310, 
311,562. 

Christie Mallowan, Agatha Mary Clarissa 
Miller, xvii. 

Chu Shih-Chieh (= Zhū Shìjié, Zhū 
Sdngting; RHEB, RAE), 53, 59, 70. 

Chung Graham, Fan Rong King 
GERIA), 514. 

Chung, Kai Lai ($F), 105. 

CI: The comparison indicator of MIX, 126, 
134, 142, 211, 213, 228. 

Circle of buffers, 218-227, 231. 

Circular definition, 263, 308, see 
Definition, circular. 

Circular linking, 273-280, 302, 355, 411, 459. 

Circular store, 240. 

Circulating shift, 135. 

CITRUS subroutine library, 457. 

Clark, Douglas Wells, 604. 

Clavius, Christopher, 159. 

Clock, for real time, 228. 

Clock, simulated, 283, 288. 

Clock, solitaire game, 377-378. 

Closed subroutine, see Subroutine. 

CMath: Concrete Mathematics, a book 
by R. L. Graham, D. E. Knuth, and 
O. Patashnik, 11. 

CMP1 (compare rI1), 134, 210-211. 

CMPA (compare rA), 134, 210-211. 

CMPX (compare rX), 134, 210-211. 

COBOL language, 424—434, 457, 458. 

Codes for difficulty of exercises, xv—xvii. 

Coding: Synonym for “programming”, but 
with even less prestige. 

Coefficient extraction, 92. 

Cofactor of element in square matrix: 
Determinant of the matrix obtained 
by replacing this element by unity and 
replacing all other elements in the same 
row or column by zero, 37, 381. 

Coffman, Edward Grady, Jr., 450-451. 

Cohen, Jacques, 421, 461, 614. 
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Coin tossing, 101—102. 
tail of distribution, 106. 

Collins, George Edwin, 461. 

Combinations of n objects taken k at a 
time, 52-53, 69, 94. 

with repetitions permitted, 73, 92, 
95, 386, 388. 
with restricted repetitions, 95. 

Combinatorial matrix, 37-38, 589. 

Combinatorial number system, 73, 560. 

Comfort, Webb T., 461. 

COMIT language, 461. 

Command: Synonym for “instruction” 

Comments, 2-3. 

in assembly language, 145, 149. 

Commutative law, 165. 

Comp. J.: The Computer Journal, a 
publication of the British Computer 
Society since 1958. 

Compacting memory, 423, 439, 449, 
452, 455. 

Comparability, 270. 

Comparison indicator of MIX, 126, 134, 
142, 211, 213, 228. 

Comparison operators of MIX, 134, 210-211. 

Compiler: A program that translates 
computer languages. 

algorithms especially for use in, 360, 
424-434, 556. 

Complete binary tree, 401, 405, 563. 

Complete t-ary tree, 401—402. 

Complex conjugate, 21. 

Complex number, 21. 

Composition of permutations, see 
Multiplication of permutations. 

Compound interest, 23—24. 

Compression of messages, 407. 

Computational error, 24—26. 

Computational method, 5, 7-8. 

Compute: To process data. 

Computer: A data processor. 

Computer language, see Assembly language, 
Machine language, Programming 
language. 

CON (constant), 149-150, 155. 

Concatenation of strings, 274. 

Concave function, 406. 

Conditional expression, 460, 624. 

Congruence, 40—42. 

Connected directed graph, 372. 

strongly, 372, 377. 

Connected graph, 363. 

Conservative law, 170, see Kirchhoff’s law. 

Constants in assembly language, 
149-150, 155. 

Construction of trees, 340-341, 343, 
428-429. 

CONTENTS, 127, 235-236. 

Context-free grammar, 539. 

Continuants, 600-601. 


Continued fractions, 498, 565. 
Continuous simulation, 282, 298. 
Convergence: An infinite sequence (Xn) 
converges if it approaches a limit as 
n approaches infinity; an infinite sum 
or product is said to “converge” or 
to “exist” if it has a value according 
to the conventions of mathematical 
calculus; see Eq. 1.2.3-(3). 
absolute, 29. 
of power series, 87, 396. 

Conversion operations of MIX, 138. 

Convex function, 404, 596. 

Convolution of probability distributions: 
The distribution obtained by adding 
two independent variables, 103. 

Conway, Melvin Edward, 151, 229. 

Conway, John Horton, 19, 80, 273, 408, 600. 

Cook, Theodore Andrea, 81. 

Copy a data structure: To duplicate a 
structured object by producing another 
distinct object that has the same data 
values and structural relationships. 

binary tree, 329-330, 333, 347. 
linear list, 279. 

List, 423. 

two-dimensional linked list, 306. 

Copying and compacting, 421. 

Corless, Robert Malcolm, 395. 

Coroutines, 193-200, 222-223, 283-296, 320. 

history, 229. 
linkage, 194, 200, 223, 291. 

Correspondence between binary trees and 
forests, 334-335, 346. 

Cousins, 317. 

Coxeter, Harold Scott MacDonald, 80, 
162, 407, 408. 

Crain, Steven Paul, 613. 

Crelle: Journal fiir die reine und angewandte 
Mathematik, an international journal 
founded by A. L. Crelle in 1826. 

Crelle, August Leopold, 58, 634. 

Critical path time, 217. 

Crossword puzzle, 163. 

Crowe, Donald Warren, 601. 

Cullen, Christopher, 75. 

Cumulants of a probability distribution, 
103-106. 

Cycle: Path from a vertex to itself. 

detection of, 271. 
fundamental, 366-370, 377. 
Hamiltonian, 374, 379. 
in directed graph, 363. 
in graph, 363. 
in permutation, 164-167, 176-178, 
182-184. 
in random permutation, 179-184. 
notation for permutations, 164-167, 
172-175, 182. 
oriented, in directed graph, 372. 
singleton, in permutation, 164, 171, 
180-181. 
Cycle lemma, 593. 
Cyclic shift, 185. 


d’Imperio, Mary Evelyn, 462. 

Dahl, Ole-Johan, 229, 230, 461, 462. 

Dahm, David Michael, 433, 434. 

Data (originally the plural of “datum” 
but now used collectively as singular 
or plural, like “information”): 
Representation in a precise, formalized 
language of some facts or concepts, 
often numeric or alphabetic values, 
to facilitate manipulation by a 
computational method, 215. 

packed, 128, 158. 

Data organization: A way to represent 
information in a data structure, 
together with algorithms that access 
and/or modify the structure. 

Data structure: A table of data including 
structural relationships, 232—465. 

linear list structures, 238-298. 
List structures, 408—423. 
multilinked structures, 424—434. 
orthogonal lists, 298-307, 424—434. 
tree structures, 308—408. 

Daughter, in a tree structure, 311. 

David, Florence Nightingale, 66. 

Davies, David Julian Meredith, 445. 

Davis, Philip Jacob, 50. 

Dawson, Reed, 584. 

de Bruijn, Nicolaas Govert, 121, 122, 375, 
379, 380, 478, 504, 543, 565. 

de Moivre, Abraham, 74, 83, 87, 106, 

182, 474, 536. 

De Morgan, Augustus, 17. 

Deallocation, see Liberation. 

Debugging: Detecting and removing 
bugs (errors), 192-193, 201, 257, 

297, 413, 556. 

DEC1 (decrease rI1), 134, 210. 

DECA (decrease rA), 134, 210. 

Decimal computer: A computer that 
manipulates numbers primarily in the 
decimal (radix ten) number system. 

Decimal number system, 21, 619. 

DECX (decrease rX), 134, 210. 

Defined symbol, an assembly language, 153. 

Definition, circular, see Circular definition. 

Degree, of node in tree, 308, 317, 377. 

of vertex in directed graph, 372. 

Deletion of a node: Removing it from a 
data structure and possibly returning 
it to available storage. 

from available space list, see Reservation. 
from deque, 251, 297. 


from doubly linked list, 281, 290-291, 297. 


from doubly linked ring structure, 358. 

from linear list, 239. 

from linked list, 236, 255, 276, 305. 

from queue, 242, 244-245, 254, 261, 
265, 273-274. 

from stack, 241, 242, 244-245, 247, 254, 
259, 269, 273-274, 278, 458. 

from tree, 358. 

from two-dimensional list, 305. 
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Demuth, Howard B., 120. 
Depth-first search, 578, 581. 
Depth of node in a tree, see Level. 
Deque: Double-ended queue, 239-243, 269. 
deletion from, 251, 297. 
input-restricted, 239-243, 416. 
insertion into, 251, 297. 
linked allocation, 280, 297. 
output-restricted, 239-243, 269, 274. 
sequential allocation, 251. 
Derangements, 180, 183. 
Derivative, 90, 338. 
Dershowitz, Nachum (\721W17 DN)), 
518, 588, 598. 
Descendant, in a tree structure, 311, 
348, 562. 
Determinant of a square matrix, 37-39, 
81, 378-379, 382. 
Deuel, Phillip DeVere, Jr., 556. 
Deutsch, Laurence Peter, 418, 421, 422. 
Dewar, Robert Berriedale Keith, 614. 
Dewey, Melville (= Melvil) Louis Kossuth, 
notation for binary trees (due to 
Galton), 317, 331, 346, 405. 
notation for trees, 313, 317, 382-383, 460. 
Diaconis, Persi Warren, 491. 
Diagonals of polygons, 408. 
Diagrams of structural information, 
234, 279. 
before-and-after, 260-261, 278, 281, 571. 
binary trees, 312, 318, 563. 
List structures, 315-317, 408-409. 
tree structures, 309-315, 337, 346, 
349, 460, 465. 
Dickman, Karl Daniel, 525. 
Dickson, Leonard Eugene, 81, 484. 
Dictionaries of English, 1-2, 215-216. 
Differences of polynomials, 64. 
Differentiation, 90, 338-347, 459. 
chain rule for, 52. 
Digamma function w(z), 44, 75, 493. 
Digit: One of the symbols used in radix 
notation; usually a decimal digit, one 
of the symbols 0, 1, ..., or 9. 
Digraphs, 372, see Directed graphs. 
Dijkstra, Edsger Wybe, 17, 191, 230, 231, 
240, 459, 462, 545, 580, 605. 
d’Imperio, Mary Evelyn, 462. 
Directed graphs, 372—374, 422. 
as flow charts, 364-365, 377. 
balanced, 374-375. 
connected, 363. 
regular, 379. 
strongly connected, 372, 377. 
Discrete system simulation, 203, 282-298. 
synchronous, 282, 298. 
Disjoint sets: Sets with no common 
elements. 
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Disk files, 136-137, 435, 463. 
Disk input, buffered, 228. 


Disposal, see Garbage collection, Liberation. 


Dissection of a polygon, 408. 
Distributed-fit method of storage allocation, 
450-451, 456. 


Distribution: A specification of probabilities 


that govern the value of a random 
variable. 

binomial, 101-102. 

negative binomial, 107. 

normal, 104-106, 122. 

Poisson, 106, 502, 524. 

tails of, 104, 106-107. 

uniform, 102, 253, 446. 

Distributive law, 28, 37, 42, 598. 

DIV (divide), 131-133, 139, 208. 

Divergent series, 28, 75. 

Division converted to multiplication, 
516-518. 

Divisor: x is a divisor of y if ymoda = 0 
and x > 0; it is a proper divisor if it is 
a divisor such that 1 < a < y. 

Dixon, Alfred Cardew, 490. 

Dixon, Robert Dan, 509. 

DLINK: Link downward, 409, 411. 

Dodgson, Charles Lutwidge, see Carroll. 

Doolittle, Myrick Hascall, 303. 

Doran, Robert William, 229. 

Double generating function: A generating 
function of two variables, 94, 396, 
405, 537-539. 

Double order for traversing trees, 332, 
333, 564. 

Doubly linked lists, 280-281, 288-291, 
297-298, 357, 411, 441, 443, 452, 459. 

compared to singly linked, 281, 298. 

Dougall, John, 490. 

Doyle, Arthur Ignatius Conan, 465. 

Drum memory, 136-137, 457. 

Dull, Brutus Cyclops, 111. 

Dummy variable, 27. 

Dunlap, James Robert, 457. 

Durand, Bruno, 586. 

Dutka, Jacques, 50, 65. 

Dvoretzky, Aryeh (??87197 MIN), 593. 

Dwyer, Barry, 567. 

Dynamic storage allocation, 246-254, 
256-259, 413-414, 435-456. 

history, 457—458, 461—462. 
running time estimates, 449. 
Dynastic order, 336, see Preorder. 

DYSEAC computer, 231. 


e (base of natural logarithms), 23, 
619-620, 626. 

Earley, Jackson Clark, 462. 

Easter date, 159-160. 

Edelman, Paul Henry, 598. 

Edge in graph, 363. 


Edwards, Daniel James, 423. 
Effective algorithm, 6, 8, 9. 
Egorychev, Georgii Petrovich (Eropsrues, 
Teopruit Ierposns), 499. 
Eisele, Peter, 480. 
Eisenstein, Ferdinand Gotthold Max, 479. 
Elementary symmetric functions, 38, 
94, 497. 
Elevator (lift) system, 282-298. 
Ellen [Fich] Whyne, Faith, 523. 
Ellipses (--- or ...), 27, 34, 46. 
Embedding of partial order into linear order, 
262, see Topological sorting. 
Embedding of tree in another tree, 348, 386. 
Empty list, 244-245, 247, 258, 260-261, 
273-275, 278, 280, 540, 546. 
Emulation, 202. 
END (end of program), 151, 156, 296. 
End of file, 216, 227. 
Endorder, see Postorder. 
Engles, Robert William, 462. 
English letter frequencies, 159. 
ENN1 (enter negative into rI1), 133, 210. 
ENNA (enter negative into rA), 133, 210. 
ENNX (enter negative into rX), 133, 210. 
ENT1 (enter into rI1), 133, 210. 
ENTA (enter into rA), 133, 210. 
Entity, 233, see Node. 
Entrances to subroutines, 186-191. 
multiple, 189. 
ENTX (enter into rX), 133, 210. 
Enumeration of subtrees, 378-379. 
Enumeration of tree structures, 386-399. 
history, 406—407. 
Epictetus of Hierapolis ("Extxtyto¢ 
6 “IepardAews), 1. 
EQU (equivalent to), 146, 149, 155. 
Equivalence algorithm, 360-361, 578, 581. 
Equivalence classes, 354. 
Equivalence declarations, 360. 
Equivalence relations, 353-355, 487. 
Equivalent algorithms, 467. 
Equivalent binary trees, 328-329. 
Equivalent forests, 346. 
Equivalent Lists, 423. 
Equivalent of a MIXAL symbol, 156. 
Equivalent trees, 346. 
Erase a data structure: To return all of its 
nodes to available storage. 
linear list, 273, 274, 279. 
List, 413-414. 
right-threaded binary tree, 333. 
Erdélyi, Artúr (= Arthur), 399. 
Erdwinn, Joel Dyne, 229. 
Errors, avoiding, 260—261, 556. 
computational, 305. 
detection of, 192-193, 201, 257, 297, 413. 
Etherington, Ivor Malcolm Haddon, 399. 
Ettingshausen, Andreas von, 54. 
Etymology, 1-2. 


Euclid (EdxAeténc), 2, 5. 
algorithm for gcd, 2-9, 80. 
algorithm for gcd, extended, 13-14, 42. 
Euclidean domains, 468. 
Euler, Leonhard (Eitneps, Jleonapas 
= Ditnep, Jleonapaz), 49, 50, 52, 
57, 75, 76, 87, 111, 374, 407, 472, 
493, 496, 536, 600. 
constant y, 75, 107, 114, 619-620. 
summation formula, 111-116, 120, 123. 
theorem, 42. 
totient function y(n), 42, 184. 
Eulerian trails in a directed graph, 
374-376, 379, 584. 
enumeration of, 380. 
Eulerian numbers, second-order, 506. 
Evaluation of powers, 509. 
Evaluation of tree functions, 351, 361. 
Evans, Arthur, Jr., 202. 
Even permutations, 185. 
Exchange operation (+), 3, 182, 274. 
Exclusive or, 442, 455, 553. 
Execution time, methods for studying, 
96, 170-172. 
for MIX instructions, 138-141. 
Exercises, notes on, xv—xvii, 284. 
Exit: The place where control leaves 
a routine. 
Exits from subroutines, 186-191. 
Expected value of a random variable: The 
average or “mean” value, 98, 103. 
from a generating function, 100-103. 
Exponential generating function for (an): 
X anz”/n!, 89. 
Exponential integral F; (x), 498. 
Exponents, laws of, 22, 25, 52. 
Extended binary trees, 399—406. 
Extended Euclidean algorithm, 13-14, 42. 
Extendible matrix, 307. 
External nodes, 400—405. 
External path length, 400, 405. 
Extreme and mean ratio, 80. 


Faa di Bruno, Francesco, 483. 
Factorials, 46-52, 55. 

related to gamma function, 49. 
Factorial powers, 50, 52, 67, 71, 109-110. 
FADD (floating add), 306. 
Fail-safe program, 270. 
Fallacious reasoning, 18, 111, 465. 
Falling powers, 50, 67, 69, 624. 
Family order, 350, 577. 

sequential representation of trees, 350. 
Family trees, 310-311, 317, 406, 465. 
Farber, David Jack, 461. 
Farey, John, 520. 

series, 161. 
Father, in a tree structure, 311. 
FCMP (floating compare), 507, 559. 
FDIV (floating divide), 306. 
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Ferguson, David Elton, 231, 334. 
Fermat, Pierre de, 17, 466. 
theorem, 41. 

Ferranti Mark I computer, 18. 

Feynman, Richard Phillips, 26. 

Fibonacci, Leonardo, of Pisa (= Leonardo 
filio Bonacii Pisano), 79-80, 84. 

buddy system, 455. 

generating function, 82-83. 

number system, 86, 495. 

numbers Fp: Elements of the Fibonacci 
sequence, 13, 18, 79-86. 

numbers, table of, 621. 

sequence, 13, 18, 79-86, 621. 

strings, 86. 

trees, 496. 

Fibonomial coefficients, 85, 484, 499. 

Fich, Faith Ellen Whyne, 523. 

Field: A designated portion of a set of 
data, usually consisting of contiguous 
(adjacent) symbols. 

partial, of MIX word, 126-128, 139, 
143, 207. 
within a node, 233-237. 
within a node, notations for, 235-237, 458. 

FIFO, 240, 459, see Queue. 

Fifty-percent rule, 444—445, 447, 448. 

Figurate numbers, 54. 

Filters, 198. 

Final vertex of an arc, 372. 

Fine, Nathan Jacob, 484. 

First-fit method of storage allocation, 
436-438, 453-456, 616. 

First in, first out, 240, 351, 459, 607, 
see Queue. 

Fischer, Michael John, 353. 

Fixed element of permutation, 164, 180-181. 

Fixed point arithmetic, 158. 

Flag, see Sentinel. 

Flajolet, Philippe Patrick Michel, 501, 
506, 543, 565. 

Fletcher, William, 527. 

Floating point arithmetic, 131, 306. 

operators of MIX, 131, 557-559. 

Floor function |x|, 39—41. 

Flow charts, viii, 2-3, 15-18, 364-365, 377. 

Floyd, Robert W, x, 17, 20, 422, 467, 509. 

FLPL subroutine library, 460—461. 

Flye Sainte-Marie, Camille, 584. 

FMUL (floating multiply), 306. 

FOCS: Proceedings of the IEEE Symposia 
on Foundations of Computer Science 
(1975-), formerly called the Symposia 
on Switching Circuit Theory and 
Logic Design (1960-1965), Symposia 
on Switching and Automata Theory 
(1966-1974). 

Forder, Henry George, 597. 

Forecasting, 224. 
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Forest: Zero or more trees, 309, 408, 
see Trees. 
correspondence to binary trees, 
334-335, 346. 
enumeration, 389, 594. 
index notation for, 313, 315, 317. 


Formulas, algebraic, see Algebraic formulas. 


Forstemann, Wilhelm August, 490. 
FORTRAN language, viii, 231, 233, 296, 
360, 458, 460. 
Foster, Frederic Gordon, 100. 
Fourier, Jean Baptiste Joseph, 27. 
Fractional part, 40. 
Fraenkel, Aviezri S (97399 VWIN), 251. 
Fragmentation, 439, 449, 450, 456. 
Fredman, Michael Lawrence, 514. 
Free lattice, 347. 
Free storage, see Available space. 
Free subtrees, 365-370. 
enumeration of, 378-379. 
minimum cost, 371. 
Free trees, 363-371. 
definition of, 363. 
enumeration of, 387—388, 398, 407. 
Friedman, Daniel Paul, 421. 
Frieze patterns, 407—408. 
Front of queue, 241. 
FSUB (floating subtract), 306. 
Fuchs, David Raymond, 202. 
Fukuoka, Hirobumi (48M x), 508. 
Full-word logical (bitwise) operations, 
442, 455, 510, 553. 
Fuller, John Edward, xi. 
Fundamental cycles in a graph, 
366-370, 377. 
Fundamental path, 368. 
Fundamental theorem of arithmetic, 42. 
Furch, Robert, 121. 
Future reference in MIXAL, 153, 156. 
restrictions on, 156. 


Galler, Bernard Aaron, 353. 
Galton, Francis, 562, 635. 
Games, solution of, 86, 272-273. 
Gamma function I (z), 49-52, 72, 
79, 116-119. 
incomplete, 117-122. 
Gao, Zhicheng (Aak), 565. 
Garbage collection, 257, 413-423, 438-439, 
449, 455, 461, 546, 551. 
efficiency of, 420—421. 
Gardner, Martin, 19, 80, 587. 
Garwick, Jan Vaumund, 248, 457. 
Gaskell, Robert Eugene, 86. 
Gasper, George, Jr., 490. 
Gates, William Henry, III, xi. 
Gau8 (= Gauss), Johann Friderich Carl (= 
Carl Friedrich), 49, 58, 95, 490. 
gcd: Greatest common divisor. 
Gelernter, Herbert Leo, 460. 


Generating functions, 82-84, 87-96, 243, 
386-389, 391-392, 394-399, 539, 542. 
double, 94, 396, 405, 537-539. 
for discrete probability distributions, 
99-107, 181. 
Genuys, François, 231. 
Geometric progression, sum of, 31, 88. 
Gerberich, Carl Luther, 460. 
Gill, Stanley, 229, 230, 457. 
Girard, Albert, 497. 
Glaisher, James Whitbread Lee, 504. 
Glassey, Charles Roger, 406. 
Gnedenko, Boris Vladimirovich ((HeneHko, 
Bopuc Baagumuposws), 105. 
GO button of MIX, 126, 139, 143-144, 211. 
Goldbach, Christian, 49, 407, 472. 
Goldberg, Joel, 528. 
Golden ratio, 13, 18, 21, 80, 83-86, 619-620. 
Goldman, Alan Joseph, 589. 
Goldstine, Herman Heine, 18, 229, 231. 
Golomb, Solomon Wolf, 184. 
Golumbic, Martin Charles 
(PIND DMN YA AY ONIN), 596. 
Goncharov, Vasilii Leonidovich (Tonsapos, 
Bacunmit Jleonnnosna), 501. 
Gonnet Haas, Gaston Henry, 395. 
Good, Irving John, 374, 395, 483, 584. 
Gopala (aT), 80. 
Gordon, Peter Stuart, xi. 
Gorn, Saul, 460. 
Gosper, Ralph William, Jr., 65. 
Gould, Henry Wadsworth, 58, 63, 
121, 485, 492. 
Gourdon, Xavier Richard, 525. 
Gower, John Clifford, 459. 
Grabner, Peter Johannes, 506. 
Graham, Ronald Lewis, 11, 633. 
Graphs, 363-372, 464. 
directed, see Directed graphs. 
Greatest common divisor, 2-9, 13-14, 
40, 81-82. 
Greatest integer function, see Floor function. 
Greek mathematics, 2, 19, 54, 593. 
Griswold, Ralph Edward, 461. 
Grounded wire symbol, 234. 
Griinbaum, Branko, 384, 587. 
Gustavson, Fred Gehrung, 305. 
Guy, Richard Kenneth, 19, 80, 273, 600. 


H-trees, 563. 

Haddon, Bruce Kenneth, 614. 
Hadeler, Karl-Peter Fritz, 480. 
Hageman, Louis Alfred, 586. 
Halayudha (garg), 53. 

Hald, Anders, 103. 

Hamel, Georg, 480. 

Hamilton, William Rowan, cycles, 374, 379. 
Hamlet, Prince of Denmark, 232. 
Hamming, Richard Wesley, 26. 
Hankel, Hermann, 49. 

Hansen, James Rone, 460. 


Hansen, Wilfred James, 421. 

Haralambous, Yannis (XapaAc&urove, 
Twávvne), 652. 

Harary, Frank, 407. 


Hardware-oriented algorithms, 26, 252, 611. 


Hardy, Godfrey Harold, 12, 406, 492, 520. 
Hare, David Edwin George, 395. 
Hare and hounds, see Military game. 
Harmonic numbers Hn, 75-79, 114. 
generating function, 90. 
table, 621-622. 
Harmonic series, 75, 160-161. 
Haros, Charles, 520. 
Hartmanis, Juris, 464. 
Hautus, Matheus Lodewijk Johannes, 489. 
hef, see gcd. 
Head of list, see List head. 
Heap, 435, see Pool of available nodes. 
Height of tree or forest, 565. 
Heine, Heinrich Eduard, 490. 
Hellerman, Herbert, 459. 
Hemacandra, Acarya (ATaTd FA4=z), 80. 
Henkin, Léon Albert, 17. 
Henrici, Peter Karl Eugen, 88. 
Herbert, George, xiv. 
Hermite, Charles, 49, 478. 
Herstein, Israel Nathan, 526. 
Hesse-Kassel, Louise Wilhelmine Friederike 
Karoline Auguste Julia von, 310, 311. 
Heyting, Arend, 406. 
Hilbert, David, matrix, 38. 
Hiles, John Owen, 421. 
Hill, Robert, 518. 
Hindu mathematics, 53-54, 80, 495. 
Hipparchus of Nicæa (“Innapyoc 
ô èx Nuxatac), 592. 
HLT (halt), 136, 143. 
Hoare, Charles Antony Richard, 17, 
230, 461, 462. 
Hobbes, Thomas, 652. 
Hobby, John Douglas, 652. 
Hofri, Micha (o19N n>), 515. 
Holmes, Thomas Sherlock Scott, 465. 
Holt Hopfenberg, Anatol Wolf, 460. 
Honeywell H800, 124. 
Hopcroft, John Edward, 560. 
Hopper, Grace Brewster Murray, 229. 
Horning, James Jay, 230. 
Hu, Te Chiang (4H45M), 405, 596. 
Huang Bing-Chao (HRH), 176. 
Huffman, David Albert, 402, 407. 
algorithm, 402—406. 
Hurwitz, Adolf, 44. 
binomial theorem, 399, 488. 
Hwang, Frank Kwangming (36H), 
405, 595. 
Hwang, Hsien-Kuei (H88), 66. 
Hypergeometric functions, 65. 
basic, 490. 
Hyperfactorial, 116. 
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I/O: Input or output, 215. 

Il-register of MIX, 125, 142. 

IBM 650 computer, i, 124, 230, 529. 

IBM 701 computer, 230. 

IBM 705 computer, 230. 

IBM 709 computer, 124, 529. 

IBM 7070 computer, 124. 

Ibn al-Haytham, Abū ‘Ali al-Hasan (= 
Alhazen, augll go cuall gle si), 162. 

Identity permutation, 164, 175. 

Iliffe, John Kenneth, 462. 

ILLIAC I computer, 230. 

Imaginary part of complex number, 21. 

d’Imperio, Mary Evelyn, 462. 

IN (input), 137, 215-216. 

In-degree of vertex, 372. 

in situ permutation, 9, 165, 185, 523. 

INC1 (increase rI1), 133, 210. 

INCA (increase rA), 133, 210. 

Incidence matrix, 270. 

Inclusion and exclusion principle, 181, 184. 

Inclusive ancestors and descendants, 311. 

Incomplete gamma function 7(a, x), 117-122. 

INCX (increase rX), 133, 210. 

Indentation, 312. 

Index: A number that indicates a particular 
element of an array (often called a 
“subscript”), 4, 299, 313, 315. 

Index register, 125, 127, 158, 266. 

modification of MIX instructions, 
127, 251-252. 

Index variable, 27. 

Indian mathematics, 53-54, 80, 495. 

Indirect addressing, 246, 251-252, 306. 

Induction, mathematical, 11-21, 32, 

316, 475. 
generalized, 20. 

Inductive assertions, 15-20. 

Inductive closure, 475. 

Infinite series: A sum over infinitely many 
values, 27—29, 58, 87—96. 

Infinite trees, 317, 382. 

Infinity lemma, 382-386. 

Information: The meaning associated with 
data—the facts or concepts represented 
by data; often used also in a narrower 
sense as a synonym for “data” or in a 
broader sense to include any concepts 
that can be deduced from data. 

Information structure, see Data structure. 

Ingalls, Daniel Henry Holmes, 522. 

Initial vertex of an arc, 372. 

Inorder for a binary tree, 319-323, 
330-332, 346. 

Input, 5, 215-228. 

anticipated, 159, 216, 224. 
buffering, 159, 216-228, 231. 
operators of MIX, 136-138, 215-216. 

Input-restricted deque, 239-243, 416. 
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Insertion of a node: Entering it into 
a data structure. 
into available space list, see Liberation. 
into deque, 251, 297. 
into doubly linked list, 281, 290, 297. 
into doubly linked ring structure, 358. 
into linear list, 239. 
into linked list, 235, 255, 276, 305. 
Insertion of a node (continued) 
into quadruply linked binary tree, 333. 
into queue, 242, 244-245, 254, 260, 
265, 273-274. 
into threaded binary tree, 327, 332. 
into two-dimensional list, 305. 
onto a stack, 241, 242, 244-245, 247, 254, 
258, 269, 273-274, 278, 458. 

Instruction, machine language: A code 
that, when interpreted by the circuitry 
of a computer, causes the computer 
to perform some action. 

in MIX, 127-144. 
symbolic form, 128, 144. 

INT (interrupt), 228. 

Integers, 21. 

Integration, 90. 

by parts, 77, 112-113. 
related to summation, 111-116. 

Interchange operation (+), 3, 182, 274. 

Interchanging the order of summation, 
29, 33, 35, 43. 

Interest, compound, 23-24. 

Interlock time: Delay of one part of a 
system while another part is busy 
completing some action. 

Internal nodes, 400-406. 

Internal path length, 400, 402, 405. 

Internet, iv, xvi. 

Interpreter (interpretive routine), 
200-202, 230, 340. 

Interrupt, 228. 

Intervals, notation for, 21. 

Invariants, 17. 

Inverse modulo nm, 42. 

Inverse of a matrix, 37-38, 73, 307. 


Inverse of a permutation, 106, 175-178, 182. 


Inversion problem, 63-64. 

Inversions of a permutation, 542, 557, 577. 

Inverting a linked list, 269, 279. 

I/O: Input or output, 215. 

IOC (input-output control), 137. 

IPL systems, 230, 458-459, 460—461, 552. 

Irrational radix, 86. 

Irreflexive relation, 261. 

Isaacs, Irving Martin, 601. 

Islamic mathematics, 1, 162. 

Isolated vertex, 374. 

Itai, Alon PPN oN), 534. 

Iverson, Kenneth Eugene, 33, 39, 459-460. 
convention, 32—33, 61, 103, 471. 


J-register of MIX, 125, 143, 186, 189, 
212-214. 

JiN (jump if rll negative), 135, 210. 

JiNN (jump if rI1 nonnegative), 135, 210. 

JiNP (jump if rI1 nonpositive), 135, 210. 

J1NZ (jump if rI1 nonzero), 135, 210. 

JiP (jump if rll positive), 135, 210. 

J1Z (jump if rll zero), 135, 210. 

JACM: Journal of the ACM, a publication 
of the Association for Computing 
Machinery since 1954. 

Jacob, Simon, 81. 

Jacobi, Carl Gustav Jacob, 490. 

Jacquard, Joseph Marie, 229. 

Jacques, Alain, 527. 

Jacquet, Philippe Pierre, 515. 

JAN (jump if rA negative), 135, 210. 

JANN (jump if rA nonnegative), 135, 210. 

JANP (jump if rA nonpositive), 135, 210. 

JANZ (jump if rA nonzero), 135, 210. 

JAP (jump if rA positive), 135, 210. 

Japanese mathematics, 112, 115. 

Jarden, Dov (10 11), 85, 494. 

Java language, 233. 

JAZ (jump if rA zero), 135, 210. 

JBUS (jump if busy), 137, 157, 212, 216. 

JE (jump if equal), 135, 209. 

Jeffrey, David John, 395. 

Jenkins, David Philip, 460. 

JG (jump if greater), 135, 209. 

JGE (jump if greater or equal), 135, 209. 

JL (jump if less), 135, 209. 

JLE (jump if less or equal), 135, 209. 

JMP (jump), 134, 187, 209, 288. 

JNE (jump if not equal), 135, 209. 

JNOV (jump if no overflow), 134, 142, 209. 

Jodeit, Jane Griffin, 462. 

Johnson, Lyle Robert, 459-460. 

Johnstone, Mark Stuart, 452. 

Jokes, 54, 200. 

Jones, Clifford Bryn, 18. 

Jones, Mary Whitmore, 378. 

Jonkers, Henricus (= Hans) Bernardus 
Maria, 614. 

Jordan, Marie Ennemond Camille, 388, 406. 

Josephus, Flavius, son of Matthias 
(NNN 12 WY = BrASBroe Iwonnoç 
Maz6tov), problem, 162, 184. 

JOV (jump if overflow), 134, 142, 209. 

Joyal, André, 395. 

JRED (jump if ready), 137, 222-223. 

JSJ (jump saving rJ), 134, 189, 210, 531. 

Jump operators of MIX, 134-135, 209. 

Jump trace, 214, 296, 528. 

JXN (jump if rX negative), 135, 210. 

JXNN (jump if rX nonnegative), 135, 210. 

JXNP (jump if rX nonpositive), 135, 210. 

JXNZ (jump if rX nonzero), 135, 210. 

JXP (jump if rX positive), 135, 210. 

JXZ (jump if rX zero), 135, 210. 


Kahn, Arthur Bertram, 268. 

Kahrimanian, Harry George, 459. 

Kallick, Bruce, 404. 

Kaplansky, Irving, 184, 526. 

Karamata, Jovan, 66. 

Karp, Richard Manning, 406. 

Katz, Leo, 590. 

Kaucky, Josef, 63. 

Keller, Helen Adams, 123. 

Kepler, Johannes, 80, 81. 

Kilmer, Alfred Joyce, 232. 

King, James Cornelius, 20. 

Kinkelin, Hermann, 504. 

Kirchhoff, Gustav Robert, 406, 583. 

law of conservation of flow, 97, 170-171, 
268, 278, 364-370, 380. 

Kirkman, Thomas Penyngton, 408. 

Kirschenhofer, Peter, 506. 

Klarner, David Anthony, 86. 

Kleitman, Daniel J (Isaiah Solomon), 

547, 596. 

Knopp, Konrad Hermann Theodor, 48, 498. 

Knotted lists, 459. 

Knowlton, Kenneth Charles, 462. 

Knuth, Donald Ervin (4842H), ii, iv, xi, 11, 
33, 66, 120, 193, 201, 202, 296, 297, 395, 
457, 461, 471, 484, 499, 504, 523, 525, 
565, 579, 580, 584, 592, 596, 633, 652. 

Knuth, Nancy Jill Carter (AER), x, XX. 

Kolmogorov, Andrei Nikolaevich 
(Konmoropos, Anzpeit Huxonaesn4), 
104, 105, 464. 

Kőnig, Dénes, 382, 406, 588. 

Koster, Cornelis (= Kees) Hermanus 
Antonius, 461. 

Kozelka, Robert Marvin, 544. 

Kramp, Christian, 49, 486. 

Krattenthaler, Christian, 39. 

Kreweras, Germain, 598. 

Krogdahl, Stein, 616. 

Kronecker, Leopold, delta notation, 

33, 61, 624. 

Kruskal, Joseph Bernard, Jr., 386, 588. 

Kummer, Ernst Eduard, 70. 

Kung, Hsiang Tsung (FL ## Hf), 563. 


Labeled trees, enumeration of, 389, 407. 

Labelle, Gilbert, 395. 

Lagrange (= de la Grange), Joseph 
Louis, Comte, 

inversion formula, 392, 594. 

Lamé, Gabriel, 407. 

Lamport, Leslie B., 605. 

Language: A set of strings of symbols, 
usually accompanied by conventions 
that assign a “meaning” to each string 
in the set, 5, 241, 460—461. 

machine, viii—x, 124. 

Laplace (= de la Place), Pierre Simon, 

Marquis de, 87. 
transform, 94. 
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Laplacian matrix of a graph, 583. 
Lapko, Olga Georgievna (Jlanko, Onpra 
Teopruesua), 652. 
Large programs, writing, 191-193. 
Larus, James Richard, 369. 
Last in, first out, 240, 452, 459, see Stack. 
almost, 446, 449, 455. 
Latency, 228, 457. 
Lattice: An algebraic system that 
generalizes operations like U and N. 
defined on forests, 577, 598. 
free, 347. 
Laurent, Paul Mathieu Hermann, series, 473. 
Lawson, Harold Wilbur, Jr., 433, 461. 
LCHILD field, 352-353, 359-360. 
LD1 (load rI1), 129, 208. 
LD1iN (load rI1 negative), 129, 208. 
LDA (load rA), 129, 208. 
LDAN (load rA negative), 129, 208. 
LDX (load rX), 129, 208. 
LDXN (load rX negative), 129, 208. 
Leaf of tree, 308, see Terminal node. 
Least-recently-used replacement, 452. 
Leeuwen, Jan van, 596. 
Left-child/right-sibling links, 335, 348. 
Left-sibling/right-child links, 346. 
Left subtree in binary tree, 312, 318. 
Left-to-right maximum or minimum, 
97-101, 104-106, 179. 
Legendre (= Le Gendre), Adrien Marie, 
48, 49, 51. 
symbol, 45. 
Léger, Emile, 80. 
Lehmer, Derrick Henry, 465. 
Leibniz, Gottfried Wilhelm, Freiherr 
von, 2, 51. 
Leighton, Frank Thomson, 450—451. 
Leiner, Alan Lewine, 231. 
Lenormande, Claude, 527. 
Lentin, André, 527. 
Leonardo of Pisa, 79-80, 84. 
Leroux, Pierre, 395. 
Letter frequencies in English, 159. 
Level of node in a tree, 308, 316, 317. 
Level order, 351, 564, 577. 
sequential representation, 351, 359. 
LeVeque, William Judson, 466. 
Lévy, Paul, 105. 
Levy, Silvio Vieira Ferreira, xi. 
Lexicographic order, 20, 299-300, 306, 564. 
Liberation of reserved storage, 256, 259, 
291, 413-414, 420-421, 438-442, 
443-444, 452-456. 
LIFO, 240, 459, see Stack. 
Lilius, Aloysius, 159. 
Lindstrom, Gary Edward, 567—568. 
Line printer, 136-137. 
Lineal chart, 310-311, 465. 
Linear extensions, see Topological sorting. 
Linear lists, 232, 238-307. 
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Linear ordering, 20, 262, 270. 

embedding a partial ordering into, 262, 
see Topological sorting. 

of binary trees, 333. 

of trees, 346. 

Linear probing, 451. 

Linear recurrences, 83, 88. 

Link, 233-237. 
diagram, 234. 
field, purpose, 432-433, 462. 
manipulation, avoiding errors in, 260-261. 
null, 234-235. 

Link variable, 234. 

Linked allocation of tables, 234, 254-256. 
arrays, 301-307. 
contrasted to sequential, 254-256, 

296, 433. 
history, 457—461. 
linear lists, 234-237, 264-266, 269. 
tree structures, 334, 352-357. 

Linked memory philosophy, 255, 435. 

Linking automaton, 463—464. 

Linsky, Vladislav Sergeevich (JIunckuit, 

Baagucnas Cepreesnu), 471. 

LISP language, 233, 460-461. 

LISP 2 garbage collector, 614. 

List: Ordered sequence of zero or 

more elements. 
circular, 273-280, 302, 355, 411, 459. 
doubly linked, 280-281, 288-291, 297-298, 
357, 411, 441, 443, 452, 459. 
linear, 232, 238-307. 
of available space, see Available space list. 
List (capital-List) structures, 315-316, 
408-423, 460—462. 
copying, 423. 
diagrams of, 315-317, 408—409. 
distinguished from lists, 233, 411. 
equivalence between, 423. 
notations for, 315-317, 408—409. 
representation of, 409-412, 421. 
sequential allocation, 421. 

List head, in circular lists, 275, 302-303. 

in doubly linked lists, 280-281, 
288-289, 441, 443. 

in Lists, 410, 414, 417. 

in threaded binary tree, 324, 334. 

in threaded trees, 337. 

List processing systems, 233, 412, 460—462. 

Listing, Johann Benedict, 406. 

Literal constants in MIXAL, 150, 156. 

Literate programming, 193. 

Littlewood, John Edensor, 406. 

LLINK: Link to the left, 280-281, 288-291. 
in binary trees, 318, 322, 327, 333, 459. 
in Lists, 411. 
in trees, 338, 348, 359, 380. 

LLINKT, 324. 

Lloyd, Stuart Phinney, 183, 184. 

Loading operators of MIX, 129, 139, 208. 


Loading routine, 144, 271-272. 

LOC, 235-236. 

Local symbols of MIXAL, 150-151, 157. 

Locally defined function in tree, 351, 361. 

Location counter in MIXAL, 155. 

Location field of MIXAL line, 145, 152. 

Logan, Benjamin Franklin (= Tex), Jr., 74. 

Logarithms, 22—26. 

binary, 23, 26. 
common, 23, 26. 
natural, 23, 26. 
power series, 91. 

Loop in a directed graph: Arc from a 
vertex to itself, 372. 

Loopstra, Bram Jan, 231. 

Louise Wilhelmine Friederike Karoline 
Auguste Julia von Hesse-Kassel, 
310, 311. 

Lovász, László, 491. 

Lovelace, Augusta Ada Byron King, 
Countess of, 1. 

LTAG, 322, 333, 349-350, 352, 359, 380. 

Lucas, François Édouard Anatole, 69, 
80, 81, 85, 273, 494. 

numbers Ln, 495. 

Luhn, Hans Peter, 457. 

Łukasiewicz, Jan, 338. 

Lunch counter problem, 456. 

Luo, Jianjin (F WA), 407. 

Lushbaugh, Warren Arthur, 19. 

Lynch, William Charles, 585. 


Machine language: A language that directly 
governs a computer’s actions, as 
it is interpreted by the computer’s 
circuitry, viii—x, 124. 
symbolic, 144, see Assembly language. 
MacMahon, Percy Alexander, 490, 589. 
Macro instruction: Specification of a pattern 
of instructions and/or pseudo-operators 
that may be repeated frequently 
within a program. 
Madnick, Stuart Elliot, 461. 
Magic square, 162. 
Magnetic tape, 136-137, 463. 
Mahavira, Acarya ( 
Mailloux, Barry James, 461. 
Majorization, 406. 
malloc, see Dynamic storage allocation. 
Mallat, Stéphane Georges, 629. 
Mallows, Colin Lingwood, 536. 
Margolin, Barry Herbert, 450. 
Mark I calculator (Harvard), 229. 
Mark I computer (Manchester/Ferranti), 18. 
Mark bits, 413-414. 
Marking algorithms: Algorithms that 
“mark” all of the nodes accessible from 
some given node, 271-272, 415-423. 


), 54. 


Markov (= Markoff), Andrei Andreevich 
(Mapxos, Anzpeit Aumpeesn4), 
the elder, 495. 

chain: Path taken by a Markov process. 
process, 253, 380-382. 

Markov, Andrei Andreevich (Mapxos, 
Angpeit Annpeesus), the younger, 9. 

Markowitz, Harry Max, 461. 

Markowsky, George, 80, 404. 

Martin, Alain Jean, 605. 

Martin, Johannes Jakob, 614. 

Math. Comp.: Mathematics of Computation 
(1960—), a publication of the American 
Mathematical Society since 1965; 
founded by the National Research 
Council of the National Academy 
of Sciences under the original title 
Mathematical Tables and Other Aids 
to Computation (1943-1959). 


Mathematical induction, 11—21, 32, 316, 475. 


generalized, 20. 
Matiyasevich, Yuri Vladimirovich 
(Maruacesuy, FOpuit Bnanumuposm4), 


86. 
Matrix: A two-dimensional array, 
298-299, 315. 


Cauchy, 37-38, 475. 
characteristic polynomial of, 499. 
combinatorial, 37-38, 589. 
determinant of, 37-39, 81, 378-379, 382. 
extendible, 307. 
Hilbert, 38. 
incidence, 270. 
inverse of, 37—38, 73, 307. 
multiplication, 306. 
permanent of, 51. 
representation of, 158-159, 298-307. 
singular, 307. 
sparse, 302-306. 
transpose of, 182. 
triangular, 300, 305. 
tridiagonal, 307. 
unimodular, 601. 
Vandermonde, 37-38, 475. 
Matrix (Bush), Irving Joshua, 35, 36. 
Matrix tree theorem, 378-379, 586. 
Mauchly, John William, 230. 
Maurolico, Francesco, 17. 
Maximum, algorithm to find, 96-101, 
145, 186. 
Maximum norm, 106. 
McCall’s, v. 
McCann, Anthony Paul, 614. 
McCarthy, John, 460-461. 
McEliece, Robert James, 477. 
McIlroy, Malcolm Douglas, 576, 581. 
McKeeman, William Marshall, 230. 
Mealy, George, 462. 
Mean value, see Expected value. 
Meek, Homer Vergil, 230. 
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Meggitt, John Edward, 471. 
Melville, Robert Christian, 540. 
Memory: Part of a computer system 
used to store data, 126. 
cell of, 127. 
hierarchy, 199, 421, 435, 463. 
map, 435-436. 
types of, 238. 
Merner, Jack Newton Forsythe, 229. 
Merrett, Timothy Howard, 560. 
Merrington, Maxine, 66. 
Meton of Athens (Métwv 6 “A@nvaioc), 
cycle, 160. 
METAFONT, iv, xi, 611, 652. 
METAPOST, xi, 652. 
Military game, 273, 550. 
Miller, Kenneth William, 123. 
Ming, An-T’u (AAZ[]), 407. 
Minimum path length, 399-406. 
Minimum spanning tree, 371. 
Minimum wire length, 371. 
Minsky, Marvin Lee, 423. 
Mirimanoff, Dmitri (Mupumanos, J[murpuit 
Cemënosnu), 536. 
Mirsky, Leon, 587. 
Mitchell, William Charles, 525. 
MIX computer, viii-x, 124-144. 
assembly language for, 144-157. 
extensions to, 143, 228, 251-252, 455. 
instructions, summary, 140-141. 
simulator of, 203-212. 
Mix Barrington, David Arno, 526. 
MIXAL: MIX Assembly Language, 144-157, 
235-236. 
Mixed-radix number system, 300. 
Mixture of probability distributions, 106. 
MMIX computer, 124, 187, 215, 325. 
Mock, Owen Russell, 231. 
mod, 39-40. 
modulo, 40. 
Mohammed, John Llewelyn, 527. 
Moivre, Abraham de, 74, 83, 87, 106, 
182, 474, 536. 
Moments of probability distributions, 105. 
Monitor routine, 212, see Trace routine. 
Monte Carlo method: Experiments with 
random data, 254, 445—447. 
Moon, John Wesley, 407. 
Moore School of Electrical Engineering, 230. 
Mordell, Louis Joel, 479. 
Morris, Francis Lockwood, 18, 614. 
Morris, Joseph Martin, 567. 
Morrison, Emily Kramer, 229. 
Morrison, Philip, 229. 
Moser, Leo, 66. 
Mother, in a tree structure, 311. 
Motzkin, Theodor (= Theodore) Samuel 
(PPs INW WN), 85, 593. 
MOVE, 135, 142, 193, 211. 
MOVE CORRESPONDING, 426, 430—431, 434. 
MUG: MIX User’s Group, 643. 


644 INDEX AND GLOSSARY 


MUL (multiply), 131-132, 208. 

Multilinked structures, 232, 288-289, 

357, 424-434, 458. 

Multilist representation, 301. 

Multinomial coefficients, 65, 394. 
Multinomial theorem, 65. 

Multipass algorithms, 198-200, 201-202. 
Multiple: x is a multiple of y if x = ky 

for some integer k. 

Multiple entrances to subroutines, 189. 
Multiple exits from subroutines, 190, 269. 
Multiple summation, 33-36. 
Multiple-precision arithmetic, 202. 
Multiple-precision constants, 525, 619-621. 
Multiplication of permutations, 165-167, 
172-173, 371. 

Multiplication of polynomials, 277, 280, 361. 
Multiplication of sparse matrices, 306. 
Multiplicative function, 42—43. 

Multiset: Analogous to a set, but elements 
may appear more than once. 
Multiway decisions, 158. 

Munro, James Ian, 523. 


Nagorny, Nikolai Makarovich (Haropuorit, 
Huxonait Maxaposuy), 9. 
Nahapetian, Armen, 579. 

Napier, John, Laird of Merchiston, 23. 
Narayana, Tadepalli Venkata 

(#258 3086 Todo), 598. 

Nash, Paul, 556. 

National Science Foundation, x. 

Natural correspondence between binary 
trees and forests, 334-335, 346. 
Natural logarithms, 23, 26. 

Naur, Peter, 17. 

Nearest common ancestors, 311. 
Needham, Noel Joseph Terence Montgomery 
(FBR), 59. 

Neely, Michael, 452. 

Negative: Less than zero (not zero). 
Negative binomial distribution, 107. 
Nested parentheses, 312-313, 349, 597. 
Nested sets, 312, 317. 

Nesting store, 240. 

Network: A graph together with additional 
data, such as weights on the edges 

or vertices. 

Neumann, John von (= Margittai Neumann 
János), 18, 229, 231, 457. 

Neville, Eric Harold, 591. 

Newell, Allen, 230, 457—458, 460. 
Newton, Isaac, 22, 57, 497. 

identities, 497. 

Next-fit method of storage allocation, 
448, 453, 618. 

Nicolau, Alexandru, 614. 

Nicomachus of Gerasa (Nixdyaxo¢ 

6 éx Tepdowv), 19. 

Nielsen, Norman Russell, 450. 


Nil link, see Null link. 
Niven, Ivan Morton, 87. 
Noah, son of Lamech (79? 32 NI), 310. 
Node: Basic component of data structures, 
233, 462—464. 
address of, 233. 
diagram of, 234. 
link to, 233. 
notations for fields, 235-237, 458. 
size of, 257, 299, 435, 453. 
NODE, 236. 
Node variable, 236. 
Noncrossing partitions of a polygon, 408. 
Nonnegative: Zero or positive. 
Nonnegative coefficients, 396, 501. 
NOP (no operation), 136, 142. 
Normal distribution, 104, 122. 
approximately, 105-106. 
Notations, index to, 623-627. 
ull link (A), 234-235. 
in binary trees, 322, 331. 
in diagrams, 234. 
in trees, 318. 
NUM (convert to numeric), 138. 
Number definitions, 21. 
Number system: A language for representing 
numbers. 
binary, 24-26. 
combinatorial, 73, 560. 
decimal, 21, 619. 
Fibonacci, 86, 495. 
mixed-radix, 300. 
octal, 620. 
phi, 86. 
Number theory, elementary, 40—45. 
Nygaard, Kristen, 229, 461. 


Z 


O-notation, 107-111, 118. 
O’Beirne, Thomas Hay, 518. 

Octal values of constants, 620. 
Odlyzko, Andrew Michael, 121, 565. 
Oettinger, Anthony Gervin, 460. 
Office of Naval Research, x, 230. 


Bo di (HERE, later HEFE), 


Oldenburg, Henry, 57. 

Oldham, Jeffrey David, xi. 

Omphaloskepsis, 214. 

One-address computer, 127, 350. 

One-way equalities, 108. 

One-way linkage, see Circular linkage, 

Straight linkage. 

Onodera, Rikio OMNI} 77K), 582. 

Open subroutine, 229, see Macro instruction. 

Operation code field, of MIX instruction, 127. 
of MIXAL line, 145, 155. 

Optimal search procedure, 402. 

Order of succession to the throne, 336. 

Ordered trees, 308-309, 374, see Trees. 
enumeration of, 388-389, 398, 407. 


Ordering: A transitive relation between 
objects of a set. 
lexicographic, 20, 299-300, 306. 
linear, 20, 262, 270. 
linear, of tree structures, 333, 346. 
partial, 261—262, 269-270, 346, 562, 575. 
well-, 20, 334. 
Oresme, Nicole, 22. 
Oriented binary trees, 396-397. 
Oriented cycle in a directed graph, 372. 
Oriented forests, 353-355. 
Oriented path in a directed graph, 372. 
Oriented subtrees, enumerated, 378. 
Oriented trees, 308-309, 312, 372-382. 
Oriented walk in a directed graph, 372. 
canonically represented, 390-394, 
397-398, 590-591. 
converted to ordered trees, 347. 
defined, 373. 
enumerated, 386-387, 389-395, 406. 
represented in computer, 347, 353, 377. 
with root changed, 377. 
ORIG (origin), 146, 151, 155. 
Orlin, James Berger, 584. 
Orthogonal lists, 298-307. 
Orthogonal vectors of permutations, 184. 
Otoo, Ekow Joseph, 560. 
Otter, Richard Robert, 395, 589. 
OUT (output), 137, 225. 
Out-degree of a vertex, 372. 
Output, 5, 215-228. 
buffering, 147, 216-228, 231. 
operators of MIX, 136-138. 
Output-restricted deque, 239-243, 269, 274. 
OVERFLOW, 245-251, 256-258, 268-269. 
Overflow toggle of MIX, 126, 131, 134, 
142, 144, 208, 214, 228. 
Overlapping subtrees, 326, 603. 


Packed data: Data that has been 
compressed into a small space, as when 
two or more elements are placed into 
the same cell of memory, 128, 158. 

Paging, 452. 

Pallo, Jean Marcel, 577. 

Palm tree, 581. 

Paper sizes, 318. 

Paper tape, 136-137, 229, 231. 

Parallelism, 296. 

Parameters of subroutines, 187, 189, 229. 

Parent, in a tree structure, 311, 317, 
334-335. 

in a threaded tree, 566. 

Parent links, 347, 353-355, 359-361, 
373, 377, 427-433. 

Parentheses, 312-313, 349, 597. 

Parker, Douglass Stott, Jr., 596. 

Parmelee, Richard Paine, 450. 

Parsing Machine, 201. 
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Partial field designations in MIX, 126-128, 
139, 143, 207. 
Partial fractions, 62-63, 72, 83. 
Partial ordering, 261-262, 269-270, 
346, 562, 575. 
Partitions of a set, 74, 482. 
Partitions of an integer, 12, 34, 93. 
generating function, 87, 94. 
Pascal, Blaise, 17, 53. 
triangle, 53-54, 69, 71, 73, 85, 499, see 
Binomial coefficients. 
Pass, in a program, 198-200. 
Patashnik, Oren, 11, 633. 
Path, in a graph or directed graph, 363. 
oriented, 372. 
simple, 363, 369, 372, 376. 
Path compression, 576. 
Path length of a tree structure, 399—406. 
average, 405. 
Patience (solitaire), 377-378. 
Patt, Yale Nance, 509. 
Pattern matching, 8. 
Patterns in permutations, 243. 
Pawlak, Zdzisław, 460. 
PDP-4 computer, 124. 
Peck, John Edward Lancelot, 461. 
Pedigree, 310-312, 465. 
Peirce, Charles Santiago Sanders, 593. 
Penrose, Roger, 587. 
Perfect shuffle, 184. 
Peripheral device: An I/O component of 
a computer system, 136. 
Perlis, Alan Jay, 322, 459-460. 
Permanent of a square matrix, 51. 
Permutations, 45—46, 51, 97-98, 164-185, 
242-243. 
in place, 9, 165, 185, 523. 
inverse of, 106, 175-178, 182. 
multiplication of, 165-167, 172-173, 371. 
notations for, 164. 
orthogonal vectors of, 184. 
Perrot, Jean-Francois, 527. 
Persian mathematics, 1. 
PERT network, 261-262. 
Petkovšek, Marko, 65. 
Petolino, Joseph Anthony, Jr., 516. 
Pfaff, Johann Friedrich, 486. 
Pflug, Georg Christian, 445. 
Phi (¢), 81, see Golden ratio. 
number system, 86. 
Phidias, son of Charmides (®evdtac 
Xoaputsov), 81. 
Philco $2000 computer, 124. 
Phyllotaxis, 80. 
Pi (7), 21, 619-620. 
as “random” example, 397. 
Wallis’s product for, 52, 116. 


Pingala, Acarya ( fryer), 53. 
Pile, 240. 

Pilot ACE computer, 230. 

Pipe, 198. 


Pipeline, 528. 
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Pisano, Leonardo, 79-80, 84. 

Pivot step, 302-305, 307. 

PL/I language, 433-434. 

PL/MIX language, 156. 

Plane trees, 308, see Ordered trees. 

Plex, 458. 

Poblete Olivares, Patricio Vicente, 523. 

Poincaré, Jules Henri, 491. 

Pointer, see Link. 

Pointer machines, 464. 

Poirot, Hercule, xvii. 

Poisson, Siméon Denis, distribution, 
106, 524. 

tail of, 502. 

Polish notation, 338, see Prefix notation, 
Postfix notation. 

Polonsky, Ivan Paul, 461. 

Pólya, György (= George), 17, 93, 395, 
396, 406, 407, 496. 

Polynomials, 55, 57, 64, 67, 68, 70, 108. 
addition of, 275-280, 357-359. 
Bernoulli, 44, 113-115, 503. 
differences of, 64. 
multiplication of, 277, 280, 361. 
representation of, 275-276, 280, 356-357. 

Pool of available nodes, 257, see Available 
space list. 

Pooled buffers, 224, 227. 

Pop up a stack: Delete its top element, 
241, 242, 244-245, 247, 254, 259, 
269, 273-274, 278, 458. 

Positive: Greater than zero (not zero). 

Postfix notation, 338, 352, 593. 

Posting a new item, see Insertion. 

Postorder for a binary tree, 319, 321, 
330-332, 346. 

Postorder for a tree, 336-340, 346, 348, 460. 

Postorder with degrees, representation 
of trees, 351, 361-362. 

PostScript language, 202. 

Poupard, Yves, 598. 

Power of a number, 22. 

evaluation, 509. 

Power series: A sum of the form RDO akz", 

see Generating function. 7 
convergence of, 87, 396. 
manipulation of, 118. 

Pratt, Vaughan Ronald, 45, 539, 592. 

Prefix notation, 338. 

Preorder for a binary tree, 319, 321, 
330-332, 346. 

Preorder for a tree, 336-338, 346, 348, 460. 

Preorder sequential representation of 
trees, 349, 362. 

with degrees, 359, 460. 

Prepostorder, 568. 

Prim, Robert Clay, 371. 

Prime numbers, 19, 41, 45, 47—48, 51, 
69-70, 84-85. 

algorithm to compute, 147—149. 
factorization into, 42. 


Prinz, Dietrich Giinter, 230. 

Priority queue, 435, 556, 590. 

Probability distribution: A specification of 
probabilities that govern the value of 
a random variable, 98-107. 

average (“expected”) value of, 98-103. 
variance of, 98-103. 

Probability generating function, 103. 

Procedure, see Subroutine. 

Procedure for reading this set of books, 
xii-xiv, 9. 

Prodinger, Helmut, 506. 

Profile of a program: The number of 
times each instruction is performed, 
145, 170, 214, 296, 528. 

Program: Representation of a computational 
method in some precise, formalized 
language, 5. 

Programming language: A precise, 
formalized language in which programs 
are written. 

Programs, hints for construction of, 
191-193, 296. 

Progression, arithmetic, sum of, 11, 

31-32, 56. 

Progression, geometric, sum of, 31, 88. 

Proof of algorithms, 5—6, 13-17, 321, 
361, 422, 434. 

Proof of termination, 16-17, 19-21, 386. 

Proper ancestors and descendants, 311, 562. 

Proper divisors, see Divisor. 

Property A, 586. 

Prosody, 53, 80. 

Prüfer, Ernst Paul Heinz, 407. 

Pseudo-operator: A construction in a 
programming language that is used to 
control the translation of that language 
into machine language, 146. 

Psi function 7(z), 44, 75, 493. 

Purdom, Paul Walton, Jr., 448, 450. 

Push down list, 240, see Stack. 

Push down a stack: Insert a new top 
element, 241, 242, 244-245, 247, 254, 
258, 269, 273-274, 278, 458. 


qg-nomial coefficients, 65, 73, 484, 491. 
qg-nomial theorem, 73, 494. 
Quadratic Euclidean domains, 468. 
Quadratic reciprocity law, 45. 
Quadruply linked binary tree, 333. 
Quadruply linked trees, 357. 
Quadtrees, 564. 
Qualification of names, 424—434. 
Quasiparallel processing, 296. 
Quasitoroidal tiling, 586. 
Queue, 239-243, 264-266, 459, 577, 607. 
deletion from the front, 242, 244-245, 
254, 261, 265, 273-274. 
insertion at the rear, 242, 244—245, 
254, 260, 265, 273-274. 
linked allocation, 259-261, 269, 
273-274, 288. 
sequential allocation, 244-245, 251, 
252, 254. 


Quick, Jonathan Horatio, 502. 
Quotient, 40. 


Rabbits, 79, 84. 

Rahman, Mizan (Siete aN), 490. 

Railway network, 240. 

Ramanan, Prakash Viriyur (garry 
aMuyt morir), 538. 

Ramanujan Iyengar, Srinivasa (utesfeuren 
Trio Tes Nos SquimiHrri), 12, 121, 122. 

Ramshaw, Lyle Harold, 487. 

Ramus, Christian, 71. 

Randell, Brian, 202, 450. 

Random walk, 380-381. 

Raney, George Neal, 392, 394, 593. 

Raphael, Bertram, 460. 

Rational number, 21, 25, 161. 

Raz, Yoav (39 INV), 534. 

RCA 601 computer, 124. 

Reactive process, 5. 

Read, Ronald Cedric, 565. 

Reading: Doing input, 215. 

Real number, 21. 

Real part of complex number, 21. 

Real-time garbage collection, 423. 

Reallocation of memory, 247—251, 452, 457, 
see also Compacting memory. 

Rear of queue, 241. 

Recently used bit, 452. 

Recipe, 6. 

Reciprocity formulas, 44—45. 

Recomp II computer, 124. 

Record: A set of contiguous data; see 
also Node, 233. 

Records, blocking of, 218, 225. 

Rectangular arrays, 298-307. 

Recurrence relation: A rule that defines each 
element of a sequence in terms of the 
preceding elements, 87-89. 

Recursion induction, 321, 565, 569. 

Recursive definition, 308, 312, 315, 318, 
319, 335, 346, 357. 

Recursive Lists, 316. 

Recursive use of subroutines, 191. 

Redheffer, Raymond Moos, 491. 

Reeves, Colin Morrison, 445. 

Reference, 233, see Link. 

Reference counters, 413—414, 421, 461. 

Reflection principle, 536, 593, 598. 

Reflective laws, 55, 489. 

Reflexive relation, 261, 353. 

Registers: Portions of a computer’s 
internal circuitry in which data is 
most accessible. 

of MIX, 125. 
saving and restoring contents of, 188, 
198, 213, 228. 

Regular directed graph, 379. 

Reingold, Edward Martin (791979, 
DYN yA NWN PNS), 23, 518. 
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Relation: A property that holds for certain 
sets (usually ordered pairs) of elements; 
for example, “<” is a relation defined 
on ordered pairs (x,y) of integers, 
and the property “x < y” holds if and 
only if x is less than y. 

antisymmetric, 261. 

asymmetric, 261. 

equivalence, 353-355, 487. 

irreflexive, 261. 

reflexive, 261, 353. 

symmetric, 353. 

transitive, 108, 261, 353, see Ordering. 

Relative error, 116. 

Relatively prime integers, 40. 

Releasing a buffer, 219-223, 226-227. 

Remainder, 40, 160. 

Remove from a structure, see Deletion. 

Rényi, Alfréd, 595. 

Replacement operation (4+), 3. 

Replicative function, 43—44. 

Representation (inside a computer), 

methods for choosing, 238, 424—433. 

of algebraic formulas, 337, 459. 

of binary trees, 318, 322, 327, 333-334. 

of deques, 251, 280, 297. 

of forests, 334. 

of Lists, 409-412, 421. 

of oriented trees, 347, 353, 377. 

of polynomials, 275-276, 280, 356-357. 

of queues, 244-245, 251-254, 259-261, 
269, 273-274, 288. 

of stacks, 244-254, 258, 269-270, 
273-274, 332, 417. 

of trees, 348-357, 359-362, 459-460. 

Reprogramming, 203. 

Reservation of free storage, 256-258, 291, 
436-438, 443, 452—456. 

Reversal of a string, 185. 

Reversing a list, 269, 279. 

Reversion storage, 240. 

Ribenboim, Paulo, 466. 

Rice, Stephan Oswald, 565. 

Richmond, Lawrence Bruce, 565. 

Riemann, Georg Friedrich Bernhard, zeta 
function ¢(s), 76, 504. 

Right-child/left-sibling links, 346. 

Right-sibling/left-child links, 335, 348. 

Right subtree of a binary tree, 312, 318. 

Right-threaded binary trees, 327, 332-334. 

Right-threaded trees, 338, 380. 

Right-to-left maximum or minimum, 
97-101, 104-106, 179. 

Ring structure, 355-357. 

Riordan, John, 397, 494, 595. 

RISC: Reduced Instruction Set 
Computer, 124. 

Rising factorial powers, 50, 71, 624. 

Ritchie, Dennis MacAlistair, 461. 
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RLINK: Link to the right, 280-281, 288-291. 
in binary trees, 318, 322, 327, 333, 459. 
in Lists, 409, 411. 
in trees, 338, 348-353, 359, 380. 

RLINKT, 324. 

Robinson, Raphael Mitchel, 587. 

Robson, John Michael, 448, 452, 456, 

567, 604, 616. 

Rodrigues, Benjamin Olinde, 407. 

Roes, Piet Bernard Marie, 100. 

Rogers, Leonard James, 490. 

Rokicki, Tomas Gerhard, 202. 

Roll, 240. 

Root of a directed graph, 372. 

Root of a number, 22, 25, 110. 

Root of a tree, 308, 309, 317, 465. 
changing, in an oriented tree, 377. 

Rooting a free tree, 373. 

Roots of unity, 89. 

Rosenberg, Arnold Leonard, 560. 

Rosenstiehl, Pierre, 243. 

Ross, Douglas Taylor, 450, 458, 462. 

Rotating memory devices, 228, 457. 

Rothe, Heinrich August, 63, 71, 486. 

Rounding, 41, 83, 160, 183. 

Rousseau, Cecil Clyde, 507. 

Roving pointer, 607. 

Row major order, 159, 182, 299. 

RTAG, 322, 332-334, 338, 349-351, 359, 380. 

Running time, see Execution time. 

Russell, David Lewis, 613. 

Russell, Lawford John, 202. 


Saddle point, 159. 
Salton, Gerard Anton, 351, 459. 
Sammet, Jean Elaine, 462, 574. 
Satterthwaite, Edwin Hallowell, Jr., 231. 
Saving and restoring registers, 188, 
198, 213, 228. 
Schaffer, Alejandro Alberto, 514. 
Schatzoff, Martin, 450. 
Scherk, Heinrich Ferdinand, 490. 
Schlatter, Charles Fordemwalt, 459. 
Schlatter, William Joseph, 459. 
Schlesvig-Holstein-Sgnderborg-Gliicksborg, 
Christian von, see Christian IX. 
Scholten, Carel Steven, 231, 605. 
Schoor, Amir (TW PAN), 559. 
Schorr, Herbert, 418, 422. 
Schreiber, Peter, 81. 
Schreier, Otto, 385. 
Schröder, Friedrich Wilhelm Karl Ernst, 592. 
Schwartz, Eugene Sidney, 404. 
Schwarz, Karl Hermann Amandus, 
inequality, 36. 
Schweins, Franz Ferdinand, 489. 
Schwenk, Allen John, 495. 
Schweppe, Earl Justin, 459. 
SCOPE link, 350, 362, 434. 
Scroll, 240. 


Segner, Johann Andreas von, 407, 536. 
Seki, Takakazu (FRAN), 112, 115. 
Self-modifying code, 187, 193. 
Selfridge, John Lewis, 78. 
Semaphores, 231. 
Semi-invariants of a probability distribution, 
103-106. 
Sentinel: A special value placed in a table, 
designed to be easily recognizable 
by the accompanying program, 
217-218, 276, 567. 
Sequential (consecutive) allocation of 
tables, 244. 
arrays, 158-159, 299-301, 305-307. 
contrasted to linked, 254-256, 296, 433. 
history, 457. 
linear lists, 244-254, 264-266, 325. 
tree structures, 348-352, 359-362, 433. 
Series, infinite: An infinite sum. 
Series-parallel networks, 589. 
Sets, partition of, 74, 482. 
Sha, Jichang (Y>3E4), 547. 
Shakespeare (= Shakspere), William, 
232, 466. 
Shams Baragh, Armin (Gls Guach gaji), 526. 
Shared subtrees, 326, 603. 
Shaw, John Clifford, 230, 457—458. 
Shelf, 240. 
Shephard, Geoffrey Colin, 384, 587. 
Shepp, Lawrence Alan, 183, 184. 
Shift operators of MIX, 135, 211. 
Shor, Peter Williston, 514. 
Shore, John Edward, 445, 450. 
Shylock, 466. 
Sibling, in a tree structure, 311. 
Sibling link, 334, 336, 427—433; see also 
RLINK in trees. 
SICOMP: SIAM Journal on Computing, 
published by the Society for Industrial 
and Applied Mathematics since 1972. 
Sideways addition, 131, 480. 
Sign function (sign x), 475. 
Silver, Roland Lazarus, 527. 
Similar binary trees, 327-329. 
Similar forests, 346. 
Simon, Herbert Alexander, 230, 457—458. 
Simonovits, Miklós, 505. 
Simple oriented path, 372, 376. 
Simple path, 363, 369. 
SIMSCRIPT language, 461. 
SIMULA I language, 229. 
Simulated time, 283, 288. 
Simulation: Imitiation of some process, 445. 
continuous, 282, 298. 
discrete, 203, 282-298. 
of one computer on another, 9, 202—203. 
of one computer on itself, 212—214. 
Singh, Parmanand (mara fae), 80. 
Singleton cycle of a permutation, 164, 
171, 180-181. 
Singular matrix, 307. 


Singularity of a function, 396. 

Sister, in a tree structure, 311. 

SLA (shift left rA), 135, 530. 

SLAX (shift left rAX), 135, 530. 

SLC (shift left rAX circularly), 135, 530. 

SLIP system, 460-461. 

Sloane, Neil James Alexander, 595. 

Smallest in, first out, 556. 

SNOBOL language, 461. 

SODA: Proceedings of the ACM-SIAM 
Symposia on Discrete Algorithms, 
inaugurated in 1990. 

Software: General-purpose programs 
that extend the capabilities of 
computer hardware. 

Solitaire (patience), 377-378. 

Son, in a tree structure, 311. 

Soria, Michéle, 501. 

Spanning subtrees, 365-370, 378-379. 

minimum cost, 371. 

Sparse array trick, 307. 

Sparse matrices, 302-306. 

Sparse-update memory, 298. 

Speedcoding, 230. 

Spieß, Jürgen, 91. 

Spine of a binary tree, 568. 

SRA (shift right rA), 135, 530. 

SRAX (shift right rAX), 135, 530. 

SRC (shift right rAX circularly), 135, 530. 

ST1 (store rI1), 130, 209. 

STA (store rA), 130, 209. 

Stack, 239-243, 320-321, 323, 325-326, 351, 
361, 415-416, 422, 428—429, 458-459. 

deletion (“popping”), 241, 242, 244-245, 
247, 254, 259, 269, 273-274, 278, 458. 
insertion (“pushing”), 241, 242, 244-245, 
247, 254, 258, 269, 273-274, 278, 458. 

linked allocation, 258, 269, 270, 
273-274, 332, 417. 

pointer to, 244, 258. 

sequential allocation, 244—254, 325. 

Stack permutations, 242-243, 331. 

Standard deviation of a probability 
distribution: The square root of the 
variance, an indication of how much 
a random quantity tends to deviate 
from its expected value, 98. 

Stanford University, ii, x, 296, 554. 

Stanley, Richard Peter, 407, 593, 598. 

Staudt, Karl Georg Christian von, 406. 

Staver, Tor Bøhm, 582. 

Steady state, 381-382. 

Stearns, Richard Edwin, 464. 

Steele, Guy Lewis, Jr., 605. 

Steffens, Elisabeth Francisca Maria, 605. 

Steffensen, Johan Frederik, 503. 

Stevenson, Francis Robert, 579. 

Stickelberger, Ludwig, 51. 

Stigler, Stephen Mack, 448, 450. 
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Stirling, James, 47—49, 67, 69, 73, 

87, 115, 181. 
approximation, 47, 51, 115-116. 
Stirling numbers, 66-69, 71-74, 78, 
99-100, 506, 582, 591. 
asymptotic behavior, 66. 
combinatorial interpretations, 66, 74, 179. 
duality law, 68. 
generating functions, 91. 
modulo p, 492. 
table, 66. 

STJ (store rJ), 130, 187, 209. 

STOC: Proceedings of the ACM 
Symposia on Theory of Computing, 
inaugurated in 1969. 

Stolarsky, Kenneth Barry, 495. 

Storage allocation: Choosing memory cells 
in which to store data, see Available 
space list, Dynamic storage allocation, 
Linked allocation, Sequential allocation. 

Storage mapping function: The function 
whose value is the location of an 
array node, given the indices of that 
node, 299-301, 305-307. 

Store: British word for “memory” 

Storing operators of MIX, 130, 209. 

Straight linkage, 254, 258-259, 411. 

String: A finite sequence of zero or more 
symbols from a given alphabet, 8, 86, 
185, 274, 495, see Linear lists. 

binary, 598-599. 
concatenation, 274. 
manipulation, 461, 462. 

Strong, Hovey Raymond, Jr., 560. 

Strongly connected directed graph, 372, 377. 

Structure, how to represent, 238, 
424-433, 462. 

Struik, Dirk Jan, 57, 481, 497. 

Stuart, Alan, 100. 

STX (store rX), 130, 209. 

STZ (store zero), 130, 209. 

SUB (subtract), 131-132, 208. 

Subadditive law, 616. 

Subi, Carlos Samuel, 527. 

Subroutines, 158, 186-197, 202, 206-207, 
211, 269, 279-280, 290-291, 343. 

allocation of, 271-272. 
history, 229-230. 
linkage of, 186, 229. 

Subscript, 3—4, see Index. 

Substitution operation (+), 3. 

Subtrees, 308. 

average size of, 405. 

free, enumeration of, 378-379. 
Summation, 27-39. 

by parts, 44, 76-78. 

Euler’s formula, 111-116, 120, 123. 

interchange of order, 29, 33, 35, 43. 

multiple, 33-36. 

of arithmetic progression, 11, 31-32, 56. 
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Summation (continued): 
of binomial coefficients 56-74, 
76-78, 85, 96. 
of geometric progression, 31, 88. 
of powers, 115. 
related to integration, 111-116. 
Sun SPARCstation, 652. 
Supremum: Least upper bound, 37. 
Suri, Subhash (rS 4f), 514. 
Sutherland, Ivan Edward, 459. 
Swainson, William, 330. 
Swapping buffers, 147, 159, 217—218, 225. 
Swift, Charles James, 231. 
Swift, Jonathan, 628. 
Switching table, 158, 205-206, 209, 210, 530. 
Sylvester, James Joseph, 407, 473, 583, 586. 
Symbol manipulation: A general term 
for data processing, usually applied 
to nonnumeric processes such as 
the manipulation of strings or 
algebraic formulas. 
Symbol table algorithms, 175, 426. 
Symbolic machine language, see Assembly 
language. 
Symmetric functions, 92-94, 472—473, 497. 
elementary, 38, 94, 497. 
Symmetric order for a binary tree, 
319-323, 330-332, 346. 
Symmetric relation, 353. 
Synchronous discrete simulation, 282, 298. 
System: A set of objects or processes 
that are connected to or interacting 
with each other. 
System/360 computers, 124, 189, 230. 
Szekeres, Gyérgy (= George), 595. 
Szpilrajn, Edward, 268. 


t-ary trees, 334, 405. 
enumeration of, 397, 593. 
sequential allocation, 401. 
Table-driven program, see Interpreter, 
Switching table. 
Tables, arrangement of, inside a computer, 
see Representation. 
Tables of numerical quantities, 54, 
66, 619-621. 
Tag field in tree node, 322, see LTAG, RTAG. 
Tail inequalities, 104, 106-107. 
Tamaki, Jeanne Keiko (ET), 596. 
Tamari, Dov 9N 17), born Bernhard 
Teitler, 577. 
lattice, 577, 598. 
Tape, magnetic, 136-137, 463. 
paper, 136-137, 229, 231. 
Tarjan, Robert Endre, 243, 581. 
Taylor, Brook, series, 83, 102. 
with remainder, 117. 
Temme, Nicolaas Maria, 66, 121. 
Temporary storage: Part of memory used 
to hold a value for a comparatively 


short time while other values occupy 
the registers, 191. 

Termial function, 48, 51. 

Terminal node of a tree, 308, 318, 
352, 397, 597. 

Terminology, viii, 45, 240, 311, 362, 435. 

Ternary trees, 334, 401. 

Tetrad tiling, 383-385. 

Tetrahedral arrays, 300-301, 306. 

TEX, iv, xi, 193, 202, 611, 652. 

Theory of algorithms, 7, 9. 

Theory of automata, 230, 240, 463-464. 

Thiele, Thorvald Nicolai, 103. 

Thorelli, Lars-Erik, 603, 614. 

Thornton, Charles, 322, 459—460. 

Thread an unthreaded tree, 333. 

Thread links, 422. 

Threaded binary trees, 322, 331-332, 460. 
compared to unthreaded, 326. 
insertion into, 327, 332. 
list head in, 324, 334. 

Threaded trees, 335-336, 459. 

Three-address code, 337, 459. 

Three-dots notation (--- or ...), 27, 34, 46. 

Tiling the plane, 383. 

Time taken by a program, see Execution 

time. 

Todd, John, 475. 

Todhunter, Isaac, 182. 

Tonge, Frederic McLanahan, Jr., 460. 

Top-down process, 309, 361. 

Top of stack, 241-242. 

Topological sorting, 261-271, 346, 376, 397. 

Torelli, Gabriele, 71, 488. 

Toscano, Letterio, 50. 

Toroidal tiling, 384. 

Total ordering, 270, see Linear ordering. 

Totient function y(n), 42, 184. 

Trace routine, 192, 212-214, 230-231, 296. 

Traffic signals, 161-162. 

Trails, Eulerian, in a directed graph, 

374-376, 379-380, 584. 
Transfer instruction: A “jump” instruction. 
Transitive relation, 108, 261, 353, see 
Ordering. 

Transposing a rectangular matrix, 182. 

Transposing blocks of data, 184-185. 

Transpositions: Permutations that 

interchange two elements, 3, 182, 
274, 371. 

Traversal of binary trees, 319, 459-460. 
inorder, 320, 323. 
postorder, 331-332. 
preorder, 331-332. 

Traversal of trees, 336, 459—460. 
prepostorder, 568. 

Tree function T(z), 395. 

Tree mappings, 390. 


Trees, 232, 308-423. 
binary, see Binary trees. 
comparison of different types, 
308-309, 374. 
complete, 401—402, 405, 563. 


construction of, 340-341, 343, 428-429. 


copying of, 329-330, 333, 347. 

definition of, 308, 317, 363, 373. 

deletion from, 358. 

Dewey notation for, 313, 317, 
382-383, 460. 

diagrams of, 309-315, 337, 346, 
349, 460, 465. 

disjoint, see Forest. 

embedding of, 348, 386. 

enumeration of, 386-399, 408. 

equivalent, 346. 

erasing of, 333. 

free, see Free trees. 

history, 406—407, 459—460. 

infinite, 317, 382. 

insertion into, 327, 332. 

labeled, enumeration of, 389, 407. 

linear ordering of, 346. 

linked allocation for, 334, 352-357. 

mathematical theory of, 362—408. 

ordered, 308, 374, 388-389, see Trees. 

oriented, see Oriented trees. 

quadruply linked, 357. 

representation of, 348-357, 359-362, 
459—460. 

right-threaded, 338, 380. 

sequential allocation for, 348-352, 
359-362, 433. 

similar, 346. 

t-ary, 334, 397, 401—402, 405, 593. 

ternary, 334, 401. 

threaded, 335-336, 459. 

traversal of, 336, 459—460. 

triply linked, 353, 359-360, 427—433. 

unordered, see Oriented trees. 

unrooted, 363, see Free trees. 

Triangular matrix, 300, 305. 


Triangulations of polygons, 407, 598—601. 


Tricomi, Francesco Giacomo Filippo, 
121, 122. 

Tricks versus techniques, 574-575. 

Tridiagonal matrix, 307. 

Trigonometric functions, 44, 229, 471. 

Trilling, Laurent, 461. 

Triple order for a binary tree, 567—568. 


Triply linked tree, 353, 359-360, 427—433. 


Trit, 139. 

Tritter, Alan Levi, 576. 

Tucker, Alan Curtiss, 405. 

Turing, Alan Mathison, 18, 193, 229, 
230, 459. 

machines, 8, 230, 464. 

Tutte, William Thomas, 583. 

Twain, Mark (= Clemens, Samuel 
Langhorne), 54. 
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Twigg, David William, 523. 

Two-line notation for permutations, 
164, 182. 

Two stacks, 246, 251, 253-254. 

Two-way linkage, 280-281, 411. 

Typewriter, 136-137, 231. 


Uhler, Horace Scudder, 480. 

Ullman, Jeffrey David, 560. 

Unary operator: A function of one 

variable, 337. 

UNDERFLOW, 245, 247, 259, 268-269, 274. 

niform distribution: A probability 
distribution in which every value is 
equally likely, 102, 253, 446. 

nimodular matrices, 601. 

ninitialized arrays, 307. 

nion-find algorithm, 354, 360. 

NIVAC I computer, 151, 229, 231, 480. 

NIVAC III computer, 124. 

NIVAC SS80 computer, 124. 

NIVAC 1107 computer, 124. 

NIX operating system, 198. 

nrooted trees, 363, see Free trees. 

Unusual correspondence between 
permutations, 178-179. 

Updates to memory, synchronous, 298. 

Uspensky, Vladimir Andreevich (Ycneucxuit, 
Bnagumup Angpeesus), 464. 
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van Aardenne-Ehrenfest, Tatyana, 375, 379. 

van der Waerden, Bartel Leendert, 
385-386, 587. 

van Leeuwen, Jan, 596. 

van Wijngaarden, Adriaan, 461. 

Vandermonde, Alexandre Théophile, 

59, 70, 71. 
matrix, 37-38, 475. 

Vardi, Ilan, 504. 

Variable: A quantity that may possess 
different values as a program is being 
executed, 3—4, 235. 

link or pointer, 235. 
node, 236. 
Variable-size nodes, 435—456. 
Variance of a probability distribution, 98. 
deduced from the generating function, 
100-103. 

Vauvenargues, Luc de Clapiers, Marquis 
de, xiv. 

Vectors, see Linear lists. 

Velthuis, Frans Jozef, 652. 

Vertex in a graph, 363, 372. 

isolated, 374. 

Victorius of Aquitania, 518. 

Virtual machine, 201. 

Visiting a node, 320. 

VLSI chips, 563. 

von Ettingshausen, Andreas, 54. 

von Neumann, John (= Margittai Neumann 
János), 18, 229, 231, 457. 

von Segner, Johann Andreas, 407, 536. 

von Staudt, Karl Georg Christian, 406. 
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W-value in MIXAL, 154-155. 
Wadler, Philip Lee, 605. 
Waerden, Bartel Leendert van der, 
385-386, 587. 
Wait list, see Agenda. 
Waite, William McCastline, 418, 422, 614. 
Walk, in a graph, 363. 
oriented, 372. 
random, 380-381. 
Wall, Hubert Stanley, 481. 
Wallis, John, 22, 52. 
product for m, 52, 116. 
Wang, Hao (-F}7), 383-384. 
Ward, Martin, 576. 
Waring, Edward, 78, 472. 
Warren, Don Wyman, 359. 
Watanabe, Masatoshi QB EJE), 652. 
Watson, Dan Caldwell, 251. 
Watson, George Neville, 507. 
lemma, 123. 
Watson, Henry William, 383. 
Weakest precondition, 17. 
Weber, Helmut, 461. 
Webster, Noah, dictionary, 1, 216. 
Wedderburn, Joseph Henry Maclagan, 589. 
Wegbreit, Eliot Ben, 614. 
Wegner, Peter (= Weiden, Puttilo Leonovich 
= Beityen, Ilyrruno Jleonosnu), 306. 
Weierstrass (= Weierstra8), Karl Theodor 
Wilhelm, 382. 
Weighted path length, 402-405. 
Weiland, Richard Joel, 608. 
Weizenbaum, Joseph, 414, 459-461. 
Well-ordering, 20, 334. 
Wheeler, David John, 229, 230, 457. 
Whinihan, Michael James, 86. 
Whirlwind I computer, 230. 
Whitworth, William Allen, 182. 
Wijngaarden, Adriaan van, 461. 
Wilde, Oscar Fingal O’Flahertie Wills, 422. 
Wiles, Andrew John, 466. 
Wilf, Herbert Saul, 65, 66, 93, 94, 484. 
Wilkes, Maurice Vincent, 229, 230, 457. 
Wilson, John, theorem, 51. 
Wilson, Paul Robinson, 452. 
Windley, Peter F., 523. 
Windsor, House of, 310. 
Winkler, Phyllis Astrid Benson, xi. 


We must not... 


Wirth, Niklaus Emil, 191, 461. 

Wise, David Stephen, 251, 421, 434, 605. 

Wiseman, Neil Ernest, 421. 

Wolman, Eric, 453. 

Wolontis, Vidar Michael, 230. 

Woods Berners-Lee, Mary Lee, 230. 

Woodward, Philip Mayne, 460. 

Word: An addressable unit of computer 
memory, 125-127. 

Word size, for MIX: The number of 
different values that might be stored 
in five bytes. 

Wordsworth, William, 139. 

Worst-fit method of storage allocation, 
453, 608. 

Wortman, David Barkley, 230. 

Wrench, John William, Jr., 480, 621. 

Wright, Edward Maitland, 492, 520. 

Wright, Jesse Bowdle, 359. 

Writing: Doing output, 215. 

Writing large programs, 191-193. 

Wyman, Max, 66. 

Wythoff (= Wijthoff), Willem Abraham, 80. 


X-register of MIX, 125. 

X1 computer, 231. 

XDS 920 computer, 124. 

XOR (bitwise exclusive-or), 442, 455, 553. 


Yang Hui (AV), 53. 

Yao, Andrew Chi-Chih (@k#A#}), 543. 
Yngve, Victor Huse, 461. 

Yo-yo list, 240. 

Yoder, Michael Franz, 479. 

Young, David Monaghan, Jr., 586. 


z, for complex numbers, 21, 108. 
Zabell, Sandy Lew, 491. 

Zaks, Shmuel (DPt INwWwW), 598. 
Zave, Derek Alan, 91, 614. 
Zeckendorf, Edouard, 495. 
Zedan, Hussein, 576. 

Zeilberger, Doron (197298 91717), 65. 
Zemanek (= Zemánek), Heinz, 1. 
Zeta function ¢(s, x), 44, 76, 504. 
Zhang, Linbo (FPR), 652. 
Zimmerman, Seth, 407. 

Zorn, Max, lemma, 547. 

Zuse, Konrad, 457. 


think that computation, 


that is ratiocination, has place only in numbers. 
— THOMAS HOBBES, Elementary Philosophy (1656) 
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Character code: 


00 01 02 03 04 05 06 07 08 09 


10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 


u A BC DE F G@ HI JI K LM N OP QR Ss T U 

00 1 01 2 02 2 03 10 

No operation rA rA +V rA rA- V rAX¢+ rAxV 

ADD (0:5) SUB (0:5) MUL (0:5) 
NDP CO) FADD (6) FSUB (6) FMUL (6) 

08 2 09 2 10 2 11 2 
rA¢V ril +e V ri2 e V rI3 e V 
LDA (0:5) LD1(0:5) LD2(0:5) LD3(0:5) 

16 2 17 2 18 2 19 2 
rA + -V ril = -V rI2 — -V rI + -V 
LDAN(0:5) LD1N (0:5) LD2N (0:5) LD3N (0:5) 

24 2 25 2 26 2 27 2 
M(F) rA M(F) + rIl M(F) + rI2 M(F) + rI3 
STA(0:5) ST1(0:5) ST2(0:5) ST3(0:5) 

32 2 33 2 34 Fi 35 1+T 
M(F) < rJ M(F) + 0 Unit F busy? Control, unit F 
STJ(0:2) STZ(0:5) JBUS (0) TOC(0) 

40 1 Al 1 42 1 43 1 

rA : 0, jump rll : 0, jump rI2 : 0, jump rI3 : 0, jump 

JA(+] Ji[+] J2[+] J3 [+] 
48 1 49 1 50 1 51 1 


rA + [rA]? +M 


INCA (0) DECA(1) 
ENTA(2) ENNA(3) 


ril © [rll]? +M 


INC1(0) DEC1(1) 
ENT1(2) ENN1(3) 


r12 + [rl2]? + M 


INC2(0) DEC2(1) 
ENT2(2) ENN2(3) 


r13 + [r13]? +M 


INC3(0) DEC3(1) 
ENT3(2) ENN3(3) 


56 2 


57 2 


58 2 


59 2 


CI + rA(F): V 


CMPA (0:5) 
FCMP (6) 


CI + rI1(F) : V 


CMP1(0:5) 


CI + rl2(F) : V 


CMP2 (0:5) 


CI + rI3(F) : V 


CMP3 (0:5) 


General form: 


C = operation code, (5 : 5) field of instruction 


C t 


F = op variant, (4 : 4) field of instruction 


M = address of instruction after indexing 


Description 


OP (F) 


V = M(F) = contents of F field of location M 


OP = symbolic name for operation 


(F) = normal F setting 


t = execution time; T = interlock time 


25 26 27 28 29 30 31 32 


33 34 35 36 37 38 39 40 


41 42 43 44 45 46 47 48 49 : 


vw xX ¥Y Z01 23 45 67 89 » ©) + - * f = § 
04 12 05 10 06 2 07 1+ 2F 
rA + rAX/V Special Shift M bytes Move F words 
rX + remainder NUM(0) SLA(O) SRA(1) from M to rll 
DIV(0:5) CHAR(1) SLAX(2) SRAX(3) 
MOVE(14 
FDIV(6) HLT (2) SLC(4) SRC(5) 9 
12 2 13 2 14 2 15 2 
ri4 +} V rI5 +} V rl6 + V rX V 
LD4(0:5) LD5(0:5) LD6 (0:5) LDX (0:5) 

20 2 21 2 22 2 23 2 
rl4 + -V ri5 + -V rl6 + -V rX -V 
LD4N (0:5) LD5N (0:5) LD6N (0:5) LDXN (0:5) 

28 2 29 2 30 2 31 2 
M(F) + rI4 M(F) + rI5 M(F) + rI6 M(F) + rX 
ST4(0:5) ST5(0:5) ST6 (0:5) STX (0:5) 

36 1+T 37 It 38 1 39 1 

Input, unit F Output, unit F Unit F ready? Jumps 

JMP(0) JSJ(1) 
IN(0) OUT(0) JRED (0) JOV(2) JNOV(3) 
also [*] below 

44 1 45 1 46 1 AT 1 

rI4 : 0, jump rI5 : 0, jump rI6 : 0, jump rX : 0, jump 

J4[+] J5[+] J6 [+] JX [+] 
52 1 53 1 54 1 55 1 


rl4 + [r14]? +M 


INC4(0) DEC4(1) 
ENT4(2) ENN4(3) 


rl5 + [rl5]? +M 


INC5(0) DEC5(1) 
ENT5(2) ENN5(3) 


rI6 + [r16]? +M 


INC6(0) DEC6(1) 
ENT6(2) ENN6(3) 


rX + [rX]? +M 


INCX(0) DECX(1) 
ENTX(2) ENNX(3) 


60 2 


61 2 


62 2 


63 2 


CI + rl4(F): V 


CI + rI5(F) : V 


CI + rl6(F) : V 


CI + rX(F): V 


CMP4 (0:5) CMP5 (0:5) CMP6 (0:5) CMPX (0:5) 
[*]: [+]: 
rA = register A JL(4) < N(0) 
rX = register X JE(5) = Z(1) 
rAX = registers A and X as one JG(6) > P(2) 
rli = index register 7, 1 < i <6 JGE(7) > NN(3) 
rJ = register J JNE(8) # NZ(4) 
CI = comparison indicator JLE(9) < NP(5) 


